Manage a Multipart Upload

With the multipart upload API operation, you can upload large objects in parts. You can also use the multipart upload API to copy an existing object.

Multipart upload consists of the following three substeps:
  1. You initiate the upload.
  2. You upload parts of the object.
  3. After you upload all parts of the object, you complete the upload.
After the successful completion of the multipart upload, vCloud Director Object Storage Extension builds the object using all uploaded parts and you can access the object just as you access any other objects in your buckets.

You can list all multipart uploads that are in progress and you can get a list of all the parts that you uploaded for a specific multipart upload.

When you initiate a multipart upload, vCloud Director Object Storage Extension returns an upload ID that is a unique identifier for the multipart upload. You use the ID to upload parts, to complete the upload, or to abort it.

Procedure

  1. List all multipart uploads in the bucket.
    GET https://vcloud-object-storage.example.com:8443/api/v1/s3/bucket-name?uploads
    The system returns details about all multipart uploads in the bucket.
  2. Initiate a multipart upload.
    POST https://vcloud-object-storage.example.com:8443/api/v1/s3/bucket-name/object-name?uploads
    For more information about the operation and the request body, see https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html.
    Upon a successful creation of the multipart upload, the system returns an upload ID. Note the ID, as you need it to upload parts and to complete the upload.
  3. Upload parts.
    You must include the multipart upload ID and the part number in the upload part reqruest.

    Part numbers can be any number from 1 to 10,000, inclusive. A part number uniquely identifies a part and also defines its position within the object being created. If you upload a new part using the same part number that was used with a previous part, the previously uploaded part is overwritten. Each part must be at least 5 MB, except the last part. There is no size limit on the last part of your multipart upload.

    PUT https://vcloud-object-storage.example.com:8443/api/v1/s3/bucket-name/object-name?uploadId, partNumber=number-of-the-part-you-upload
    Upon a successful upload, the system returns a part number and an ETag value. Note the two values for each uploaded part, as you need them for completing the multipart upload.
  4. List the previously uploaded parts.
    GET https://vcloud-object-storage.example.com:8443/api/v1/s3/bucket-name/object-name?encoding-type=encoding-method,max-parts=max-number-of-parts,part-number-marker=number,uploadId
    By default, the request a maximum of 1000 uploaded parts. To change the default value, use the max-parts argument.

    To select the part after which the listing begins, use the part-number-marker.

    For more information, see https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html.

  5. Complete multipart upload.
    After you successfully upload all parts of an object, use this vCloud Director Object Storage Extension S3 API request to build the object using all parts. vCloud Director Object Storage Extension concatenates all parts in ascending order by part number to create a new object.

    You must provide a complete list of the uploaded parts in the request. The operation only concatenates parts that are present in the list. For each part in the list, you must provide the part number and the ETag value, returned after that part was uploaded.

    This operation can take several minutes to complete, so vCloud Director Object Storage Extension periodically sends whitespace characters to keep the connection from timing out. Since the operation might fail after the initial 200 OK response, regularly check the response body to verify if the request succeeded.

    If you use the multipart upload API in your application, make sure that your application is prepared to retry failed requests.

    POST https://vcloud-object-storage.example.com:8443/api/v1/s3/bucket-name/object-name?uploadId