FQL v4 will be decommissioned on June 30, 2025. Ensure that you complete your migration from FQL v4 to FQL v10 by that date.

For more details, review the migration guide. Contact support@fauna.com with any questions.

import

Import JSON or CSV files into Fauna collections.

Syntax

fauna import [OPTIONS]

Description

The import command imports the contents of a JSON or CSV file, or a directory of JSON or CSV files, into a Fauna collection. If you import a directory of source files, each file is imported into a separate collection.

JSON source files must be valid JSON, and each JSON object in the file becomes a document in the target collection.

CSV source files must be comma-delimited, and each line in the CSV file becomes a document in the target collection. A CSV file must have a header line with the field names to be used in the target collection.

If your CSV file has rows with fewer fields than the number of fields in the header line, you can use the --allow-short-rows option to permit the import to proceed. Otherwise the import fails with an error. If you use the --allow-short-rows option, the documents imported from short rows do not have the missing fields.

If your CSV files has empty columns, you can use the treat-empty-csv-cells-as option to specify:

  • empty: The field exists in the imported document as an empty string.

  • null (default): The field does not exist in the imported document.

The target collection can be an existing Fauna collection or a new one. If the target collection exists and is not empty, you must use the --append option, and the new documents are added to the existing collection. If you don’t specify a target collection, Fauna Shell uses the file name of the source file as the name of the target collection.

The --path option is the path to the source file or directory of source files, and it is required. If you don’t specify any other options, Fauna Shell uses the parameters in your fauna-shell configuration file.

Floating-point values that end in .0 are converted, by JavaScript, into integers.

Billing considerations

Importing data with the import command involves billable write and compute operations. One write operation accrues for each 1kb of data written, with a minimum of 1kb per imported record, so if you import a file of 5,000 small records, you accrue 5,000 write operations. If the records are larger, for example, 5kb each, you accrue 25,000 write operations (5 each for 5,000 records).

Also, importing data into collections with covering indexes causes index updates which incur write operations.

Compute operations accrue at a rate of about one compute operation for every 20 write operations. The rate may vary depending on several factors, including record size and retries for recoverable errors.

To ensure the integrity of your data import, each source file record should include a unique identifying field. You can create a unique index on that field to ensure that no records are imported more than one time and you can query against the unique field to verify the completeness of your import.

Options

Option Description

--allow-short-rows

Optionally allows CSV files in which some rows have fewer fields than the number of fields in the header row. Note that if import finds a row which has more fields than the number of fields in the header row, the import fails with an error.

--append

Optionally appends documents to an existing collection.

--collection=<collection>

Optionally create a collection to be created. Defaults to the file name of the source file.

--db=<db>

Optionally names a child database in which to create a new target collection or append to an existing target collection. The parent database is the database associated with the secret you specify for the command.

--dry-run

Optionally performs all import operations, including record processing, and type conversions, except creating document in Fauna. This allows you to detect issues that might impact an import before writing documents to your collections.

--path=<path>

Required. This is the path to a source file or directory of source files.

Note that directories can only have files, not directories. import does not recurse into subdirectories.

--domain=<domain>

Optional Fauna server domain, that is, the hostname where Fauna is running. Defaults to db.fauna.com.

--endpoint=<name>

Optional name of the endpoint to use for the command.

--port=<number>

Optional connection port. Defaults to 8443.

--scheme=<scheme>

Optional connection scheme. Must be one of https or http. Defaults to https.

--secret=<secret>

Optional secret to use. A secret authenticates your connection to Fauna, and connects you to a database.

--timeout=<integer>

Optional connection timeout, an integer number of milliseconds. When the interval has elapsed, fauna-shell stops waiting for a response and displays an error.

The default is zero, which means that fauna-shell waits until a response is received.

--treat-empty-csv-cells-as=<empty|null>

Optionally determines how empty fields in a record should be handled:

- empty: Empty fields should occur in the imported document with an empty string.

- null (default): Empty fields should not occur in the imported document.

--type=<column>::<type>

Optionally specifies a data type for a column.

IMPORTANT: This feature is only available while importing CSV files. Type conversions are ignored for JSON and JSONL files.

You can specify a column in your source file to be imported as one of the following data types:

- bool: Converts true, t, yes, or 1 to true and all other values to false, except null which remains null.

- number: Converts strings to numbers.

- dateString: Converts an ISO-8601 or RFC-2022 date string into a time value. A best-effort is made for other date string formats, and warnings are emitted when such date conversions are performed.

- dateEpochMillis: Converts milliseconds since Unix epoch into a time value.

- dateEpochSeconds: Converts seconds since Unix epoch into a time value.

Examples

This section shows some common ways to use this command.

Import a JSON file

A file named zipcodes.json has the following:

{ "zipcode" : "01001", "city" : "AGAWAM", "pop" : 15338, "state" : "MA" }
{ "zipcode" : "01002", "city" : "CUSHMAN", "pop" : 36963, "state" : "MA" }
{ "zipcode" : "01005", "city" : "BARRE", "pop" : 4546, "state" : "MA" }
{ "zipcode" : "01007", "city" : "BELCHERTOWN", "pop" : 10579, "state" : "MA" }
{ "zipcode" : "01008", "city" : "BLANDFORD", "pop" : 1240, "state" : "MA" }

The following terminal command imports zipcodes.json:

$ fauna import --path=./zipcodes.json
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes.json to zipcodes completed. 5 rows/object imported.

In the preceding command, no --collection option is given, so Fauna Shell creates a new collection called zipcodes.

Import a JSON file and append to an existing collection

A file named zipcodes2.json has the following:

{ "zipcode" : "01010", "city" : "BRIMFIELD", "pop" : 3706, "state" : "MA" }
{ "zipcode" : "01011", "city" : "CHESTER", "pop" : 1688, "state" : "MA" }
{ "zipcode" : "01012", "city" : "CHESTERFIELD", "pop" : 177, "state" : "MA" }

The following terminal command imports zipcodes2.json and appends the documents to the existing collection zipcodes:

$ fauna import --path=./zipcodes2.json --collection=zipcodes --append
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes2.json to zipcodes completed. 3 rows/object imported.

Import a JSON file with configuration options

The following terminal command overrides any configuration options in the fauna-shell configuration file:

$ fauna import --path=./zipcodes.json --scheme=https --domain=db.us.fauna.com --port=443 --secret=secret]
Database connection established
Starting Import!
 ›   Success: Import from ./zipcodes.json to zipcodes completed. 5 rows/object imported.

Import a CSV file with type conversions

The following CSV document has three string values:

myDate,myBool,myNumber
"May 3, 2021",true,15338

To convert those string values to other types, you can use the --type option.

$ fauna import --path=./myFile.json --type=myDate::date --type=myBool::bool --type=myNumber::number]
Database connection established
Starting Import!
Warning: the string 'May 3, 2021' is not valid ISO-8601 nor RFC_2822 date. Making a best-effort translation to 'Mon May 03 2021 00:00:00 GMT-0700 (Pacific Daylight Saving Time)'
 ›   Success: Import from ./myFile.csv to myFile completed. 1 rows/object imported.

Import a directory of source files

A directory named source_files has the following files:

myJSONfile1.json
myJSONfile2.json
myCSVfile1.csv
myCSVfile2.csv

The following command imports four files and creates four new collections:

$ fauna import --path=./source_files
Database connection established
Starting Import!
 ›   Success: Import from ./source_files/myCSVfile1.csv to myCSVfile1 completed. 8 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myCSVfile2.csv to myCSVfile2 completed. 14 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myJSONfile1.json to myJSONfile1 completed. 10 rows/object imported.
Starting Import!
 ›   Success: Import from ./source_files/myJSONfile2.json to myJSONfile2 completed. 10 rows/object imported.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!