File Stream - Download in Chunks | GroveStreams

The second method looks good, in part since it is similar to what I am used to with opening and reading a file directly.

Pardon if this is redundant. For background, the file i posted is a binary image of the application code that runs on the embedded processor. I imagine a small protected portion of code on that processor that would read the file and program it into the normal run area on the processor then jump to that code for normal operation. The generic term for that is a "Bootloader" and is a common way to upgrade code on an embedded processor product. Placing the file on Grovestream, where we already have several streams monitoring normal operation solves several obstacles to the "over the air" upgrade process for a small company with limited ability to provide the servers etc to support access to the file. Probably what you meant by "...makes sense in the device world".

136

MikeMills private msg quote post Address this user

plus 1 plus 1 plus 1 0

Implemented. See start and limit parameters in the API here: https://www.grovestreams.com/developers/api.html#7

Decided not to return JSON. Easier to handle in embed code.

Note that in the API, each chunk can contain padding as per base64 specs. So, you will want to decode each chunk as it's returned.

Here's some Java code we used to test it:

[Select Code Block]

 
		String compId = "fileComp";  					//CHANGE!!!
		String streamId = "fileStream"; 				//CHANGE!!!
		String api_key = "d8a1708b-553b-399e-bc2e-f8b2036xxxxx"; //CHANGE!!!
		String fileName = "statement.pdf";      //CHANGE!!! 
		int limit = 200;
		int start = 0;
 
 
		DefaultHttpClient client = new DefaultHttpClient();
		client.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, (int)60000);
		client.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, (int)60000);
 
		byte[] fullFileBytes = new byte[0];
		boolean done = false;
		while (!done) {
			String url = String.format("http://grovestreams.com/api/comp/%s/stream/%s/feed/file/%s?api_key=%s&limit=%d&start=%d", compId, streamId, fileName, api_key, limit, start);
			HttpRequestBase meth = new HttpGet(url);
 
			HttpResponse response = client.execute(meth);
 
			if (response.getEntity() != null) {
				long responseSize = response.getEntity().getContentLength();
				if (responseSize > 0) {
					HttpEntity entity = response.getEntity();
 
					byte[] chunkBytes = Base64.getDecoder().decode(EntityUtils.toString(entity));
					fullFileBytes = Bytes.add(fullFileBytes, chunkBytes);
				} else {
					done = true;
				}
			} else {
				done = true;
			}
 
			int statusCode = response.getStatusLine().getStatusCode() ;
			if (statusCode != HttpStatus.SC_OK && statusCode != 204) {
 
				throw new RuntimeException("Http statusCode = " + statusCode);
			}
 
			start += 200;
		}
 
		if (fullFileBytes != null && fullFileBytes.length > 0) {
 
			FileUtils.writeByteArrayToFile(new File("/home/gs/Dev/testFile.pdf"), fullFileBytes);
		}
code

5 years ago Post 6 • IP flag post

1	1 crevicedweller private msg quote post Address this user	plus 1 plus 1 plus 1 0
	Base64 is OK, I prefer the string only but could work with the JSON response also.
	5 years ago	Post 5 • IP flag post

1	1 crevicedweller private msg quote post Address this user	plus 1 plus 1 plus 1 0
	This looks great. Thanks Mike. I am away from the office until Wednesday but will try this out as soon as i am back.
	5 years ago	Post 7 • IP flag post

1	1 crevicedweller private msg quote post Address this user	plus 1 plus 1 plus 1 0
	I now have demo version of code running on my target hardware that reads chunks from the file i posted. Looks like exactly what we need!
	5 years ago	Post 8 • IP flag post

1	1 crevicedweller private msg quote post Address this user	plus 1 plus 1 plus 1 0
	cancelled post
	5 years ago	Post 9 • IP flag post

1	1 crevicedweller private msg quote post Address this user	plus 1 plus 1 plus 1 0
	Cancelled post
	5 years ago	Post 10 • IP flag post

GroveStreams customer support forum

File Stream - Download in Chunks511

136	136 MikeMills private msg quote post Address this user	plus 1 plus 1 plus 1 0
	Moved from another thread: @crevicedweller: File storage looks interesting. I tried it and it works well to download the file to a PC with essentially unlimited storage. I would like to use it with an embedded processor so would need to read the file a line at at time (maybe 512 bytes per transaction), save the line to local non-volatile storage, probably on an SD card, and keep repeating until the end of the file is encountered. This would provide the essential elements of an over the air upgrade capability to the embedded processor product. Is there a way to read the file as described? If not has that been considered as a future addition?
	5 years ago	Post 1 • IP flag post

136	136 MikeMills private msg quote post Address this user	plus 1 plus 1 plus 1 0
	@crevicedweller Your suggestion makes sense in the device world. Eventually we will probably add streaming capabilities for video files, but I doubt that would help you. We could extend our API to do what you're asking. What would work for you? 1. Requesting by line number? 2. Requesting by startByteNumber and numberOfBytes
	5 years ago	Post 2 • IP flag post

136	136 MikeMills private msg quote post Address this user	plus 1 plus 1 plus 1 0
	Would you be able to convert base64 encoded bytes if we return the bytes that way? Would it be a pain if we include the base64 as part of a JSON response: [Select Code Block] `{ "streamId": "deviceCode", "data": "alskdjfal", "isBase64" : true, "time": 1380225122000 }` Or is life easier if we just return the base64 string only? alskdjfal
	5 years ago	Post 4 • IP flag post