OpenStack Object Storage (swift)

append support

Asked by Dieter P

Any plans for supporting appends to blobs?
Our users are uploading files in pieces and right now we need to buffer it locally, and once the file is finished, upload into swift.
we would like to upload into swift directly, bit by bit.

I know something like this is possible (uploading multiple blobs and having a manifest to combine hem all) but this assumes the multiple blobs are a certain, predetermined size. we want to append arbitrarily sized chunks.

Question information

Language:
English Edit question
Status:
Answered
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:
Revision history for this message
John Dickinson (notmyname) said : 		#1

The way to do this is with different segments tied together with a manifest file. There is no requirement in swift that the segments be the same size.

Revision history for this message
Constantine Peresypkin (constantine-q) said : 		#2

But this will lead to the "many small writes" problem, i.e. overpopulated containers.
Which in turn leads to a question:
What solution has less risk: server side append or container sharding?

Can you help with this problem?

Provide an answer of your own, or ask Dieter P for more information if necessary.

To post a message you must log in.