KEkeboola
Files

Create File Resource

This method allows you to upload a file to KBC File Storage. The upload should be done in two steps:

  1. Create a new file resource (use this API call).
  2. Use the provided credentials to actually upload the file data to the storage provider.

For a more detailed guide on the file upload, see the corresponding part of the Developers documentation.

File Storage is used for storing arbitrary files. This is used for data imported into tables and data exported from tables. Additionally, the File storage can be used for data backup and exchange between project participants. By default, all files are stored for 15 days. They can be stored permanently by using the isPermanent parameter. Files can also be tagged to categorize file uploads. When the X-KBC-RunId header is provided, it is stored as the file runId.

Internally, all files are stored encrypted and are private. When you access the file through the API, a URL with one hour expiration is generated.

Large files may be split into slices (chunks). A sliced file must have the isSliced parameter set to true. In that case the provided key (blob) points to a manifest file. The manifest file contains the actual list of uploaded chunks. When uploading a sliced file, you have to use the provided key as a prefix for chunks and generate the manifest file with the list of uploaded chunks. See the Developers documentation for a detailed guide on sliced files.

The accepted request attributes depend on the file storage provider of the project (S3FilePrepareRequest for AWS, AbsFilePrepareRequest for Azure, GcsFilePrepareRequest for GCP). The response contains provider-specific upload parameters: uploadParams (AWS), absUploadParams (Azure) or gcsUploadParams (GCP).

post/v2/storage/files/prepare

Request

Request body

OR
OR

Example request

{
  "name": "someFile.csv",
  "contentType": "text/csv",
  "tags": [
    "tag1",
    "tag2"
  ],
  "sizeBytes": 12345
}

Response

Created file resource including provider-specific upload parameters.

idinteger required

Identifier of the file.

createdstring required

Creation time of the file, ISO 8601 with a +hhmm offset.

isPublicboolean required

Whether the file is publicly accessible via a direct URL.

isSlicedboolean required

Whether the file consists of multiple slices (chunks) described by a manifest file.

isEncryptedboolean required

Whether the file content is encrypted in the storage.

namestring required

Sanitized file name.

fileTypestring required

Human readable file type derived from the file name.

urlstring required

Pre-signed download URL of the file. The URL expires after one hour.

provider'aws' | 'azure' | 'gcp' required

File storage provider.

regionstring required

Region of the file storage.

sizeBytesinteger nullable required

File size in bytes, null when not known.

tagsstring[] required

Tags assigned to the file.

maxAgeDaysinteger nullable required

Number of days the file is kept before it expires. Null for permanent files.

runIdstring nullable required

Run ID taken from the X-KBC-RunId header when the file was created.

runIdsstring[] required

All run IDs derived from the hierarchical run ID.

Example response

{
  "id": 236518631,
  "created": "2017-02-13T12:01:05+0100",
  "isEncrypted": true,
  "name": "somefile.csv",
  "fileType": "csv",
  "url": "https://s3.amazonaws.com/kbc-sapi-files/exp-180/578/files/2017/02/14/236518630.somefile.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Signature=374...a4c",
  "provider": "aws",
  "region": "us-east-1",
  "sizeBytes": 12345,
  "tags": [
    "tag1",
    "tag2"
  ],
  "maxAgeDays": 15,
  "runIds": [],
  "creatorToken": {
    "id": 27978,
    "description": "dev@keboola.com"
  },
  "uploadParams": {
    "key": "exp-180/578/files/2017/02/14/236518630.somefile.csv",
    "bucket": "kbc-sapi-files",
    "acl": "private",
    "credentials": {
      "AccessKeyId": "ASI...D2Q",
      "SecretAccessKey": "H+/...BYy",
      "SessionToken": "Ago...sUF",
      "Expiration": "2017-02-14T21:34:46+00:00"
    },
    "x-amz-server-side-encryption": "AES256"
  },
  "absUploadParams": {
    "blobName": "somefile.csv",
    "accountName": "kbcfshc7chguaeh2km",
    "container": "exp-15-files-4516-27364641-2022-11-09",
    "absCredentials": {
      "SASConnectionString": "BlobEndpoint=https://kbcfshc7chguaeh2km.blob.core.windows.net;SharedAccessSignature=sv=2017-11-09&sr=c&sp=rwl&sig=x6csI...",
      "expiration": "2017-02-13T12:01:05+0100"
    }
  },
  "gcsUploadParams": {
    "projectId": "ci-connection-361807",
    "bucket": "kbc-ci-connection-files-storage",
    "key": "exp-15/23058/files/2022/11/09/851783.somefile.csv",
    "access_token": "ya29.dr...",
    "expires_in": 3599,
    "token_type": "Bearer"
  }
}

Changes

No recorded changes to this operation across all 1 revision of this API.