开云体育

ctrl + shift + ? for shortcuts
© 2025 Groups.io
  1. Crestron
  2. Topics

http request chunking

 

From the research I've done on this problem I don't believe there is a solution from the Crestron side but there are people here that know a great deal more about networking than I do.?? I apologize for what will probably some abuse in my description of networking terminology.

I've run into this problem with products from 2 manufacturers. The common denominator is that both of these products use a software component named LwIP as part of their http interfaces. The issue is that LwIP has a software bug and if an http request is too big and gets broken up into multiple chunks the LwIP software component is unable to properly put them back together and the http request fails.? Normally the size that determines if a request gets broken into multiple chunks is determined by the network's MTU setting.? It is typically set to 1500.

Here is the issue.? In trying to write Crestron drivers for these products and I've found that a Crestron processor doesn't seem to obey the MTU setting for the network and breaks apart much smaller http requests into multiple chunks, which then fail.

Does anyone know of any way I can fix this on the Crestron side of things and keep the Crestron processor from breaking up http requests that are smaller than the network MTU setting into multiple chunks.

For those interested, here is a thread on the issue with LwIP



You have to keep clicking on the [Next in Thread] link to work your way through the discussion

Thanks in advance for the help

Jay
 

Jay,

The link bounces between TCP Fragmentation and HTTP Chunking. ?I’m assuming the underlying problem is HTTP Chunking. ?Crestron’s implementation of the HTTP Client chokes on chunks (and other things too). ?When I had to talk to an HTTPS server that chunked responses (it used chunks to stream changes) I ended up writing an implementation that used a SecureTcpClient and then handled the chunks on my own.

Is your problem sending data to a device running lwIP or receiving data from it?

Is lwIP advertising as HTTP/1.0 or 1.1? ?1.0 doesn’t support chunks.
 

开云体育

Thanks.

My understanding, and again, I don't claim to be a networking expert, is that this is a chunking problem.? There are no issues when the device sends messages to the Crestron system; only when the Crestron system sends messages to the device.? My understanding from the manufacturer is that the device is that the device supports http 1.1.

Thanks again

Jay

On 9/20/2023 7:16 AM, markkohlmann wrote:

Jay,

The link bounces between TCP Fragmentation and HTTP Chunking. ?I’m assuming the underlying problem is HTTP Chunking. ?Crestron’s implementation of the HTTP Client chokes on chunks (and other things too). ?When I had to talk to an HTTPS server that chunked responses (it used chunks to stream changes) I ended up writing an implementation that used a SecureTcpClient and then handled the chunks on my own.

Is your problem sending data to a device running lwIP or receiving data from it?

Is lwIP advertising as HTTP/1.0 or 1.1? ?1.0 doesn’t support chunks.

 

I ran into this with a display driver I was creating a couple years ago. I was never able to make things work properly. Essentially the device I was working with would send all responses chunked. I worked with several folks in ATSG on this and the end result was that it was an issue with the device and no effective way to work around receiving chunked data. I ran it up the chain with the display manufacturer and they ended up changing things on their end to fix it.
 

开云体育

Thanks Alex.

I'm probably in the same boat.? But in my case, it is the Crestron system that is sending messages chunked and causing the issue.? I'm just hoping there is a setting, or possibly a way in C#, to force the Crestron system not to do this.

Thanks again

Jay

On 9/21/2023 8:44 AM, Alex Teague wrote:

I ran into this with a display driver I was creating a couple years ago. I was never able to make things work properly. Essentially the device I was working with would send all responses chunked. I worked with several folks in ATSG on this and the end result was that it was an issue with the device and no effective way to work around receiving chunked data. I ran it up the chain with the display manufacturer and they ended up changing things on their end to fix it.

 

This is not HTTP Chunked Transfer Encoding, this is a problem with the device expecting the entire request to appear in the first packet received.? This is never a safe assumption with TCP connections - you should expect anything from a MTU-sized packet to one byte per packet.? Unfortunately, not all devices are programmed correctly.? Jay are you using Crestron's HTTPClient to send the request?? Have you tried using a string in SIMPL+ to blast it all out in one chunk?? As long as it's smaller than the MTU I would expect it to go out in a single packet (though that's never guaranteed).


On Thu, Sep 21, 2023 at 10:55?AM jbasen <jay.m.basen@...> wrote:

Thanks Alex.

I'm probably in the same boat.? But in my case, it is the Crestron system that is sending messages chunked and causing the issue.? I'm just hoping there is a setting, or possibly a way in C#, to force the Crestron system not to do this.

Thanks again

Jay

On 9/21/2023 8:44 AM, Alex Teague wrote:
I ran into this with a display driver I was creating a couple years ago. I was never able to make things work properly. Essentially the device I was working with would send all responses chunked. I worked with several folks in ATSG on this and the end result was that it was an issue with the device and no effective way to work around receiving chunked data. I ran it up the chain with the display manufacturer and they ended up changing things on their end to fix it.

 

开云体育

Thanks Jeremy.

The issue is a known problem in the LwIP driver used by the device manufacturer.?? That being said, since I have no control over fixing the device, I was hoping for a way to force the Crestron system to send things in one chunk.? I am using HttpClient to send everything.? I hadn't thought about trying to move strings created in S# back to S+, even Simpl, to send them from those levels.? I figured Simpl and S+ were going to use the same underlying http driver software so there wouldn't be a difference.?

At this point the problem is moot.? Because of the issue I had to abandon writing a driver for the product.

Thanks again

Jay

On 10/6/2023 1:21 PM, Jeremy Weatherford wrote:

This is not HTTP Chunked Transfer Encoding, this is a problem with the device expecting the entire request to appear in the first packet received.? This is never a safe assumption with TCP connections - you should expect anything from a MTU-sized packet to one byte per packet.? Unfortunately, not all devices are programmed correctly.? Jay are you using Crestron's HTTPClient to send the request?? Have you tried using a string in SIMPL+ to blast it all out in one chunk?? As long as it's smaller than the MTU I would expect it to go out in a single packet (though that's never guaranteed).

On Thu, Sep 21, 2023 at 10:55?AM jbasen <jay.m.basen@...> wrote:

Thanks Alex.

I'm probably in the same boat.? But in my case, it is the Crestron system that is sending messages chunked and causing the issue.? I'm just hoping there is a setting, or possibly a way in C#, to force the Crestron system not to do this.

Thanks again

Jay

On 9/21/2023 8:44 AM, Alex Teague wrote:
I ran into this with a display driver I was creating a couple years ago. I was never able to make things work properly. Essentially the device I was working with would send all responses chunked. I worked with several folks in ATSG on this and the end result was that it was an issue with the device and no effective way to work around receiving chunked data. I ran it up the chain with the display manufacturer and they ended up changing things on their end to fix it.

1 - 7 of 7