[From nobody Wed Sep 13 19:57:05 2017
From: "Jaradeh, Fares" <fares.jaradeh@nuance.com>
To: "nginx-devel@nginx.org" <nginx-devel@nginx.org>
Subject: Preserving Proxied server original Chunk sizes
Thread-Topic: Preserving Proxied server original Chunk sizes
Thread-Index: AdMsv8TFO4j9vW5QTrGzEeWO6uMq4wAA0hZAAAAtv2A=
Date: Wed, 13 Sep 2017 19:09:19 +0000
Message-ID: <BLUPR05MB306826F771A3374E484C86C9B6E0@BLUPR05MB306.namprd05.prod.outlook.com>
References: <BLUPR05MB3069DFB965993014C5C9A029B6E0@BLUPR05MB306.namprd05.prod.outlook.com>
<BLUPR05MB30623CDBD286D2001F9F1999B6E0@BLUPR05MB306.namprd05.prod.outlook.com>
In-Reply-To: <BLUPR05MB30623CDBD286D2001F9F1999B6E0@BLUPR05MB306.namprd05.prod.outlook.com>
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
X-MS-Exchange-Organization-RecordReviewCfmType: 0
x-ms-exchange-organization-originalclientipaddress: 192.40.239.178
x-ms-exchange-organization-originalserveripaddress: 10.141.23.148
x-ms-exchange-organization-submissionquotaskipped: False
Content-Type: multipart/alternative;
boundary="_000_BLUPR05MB306826F771A3374E484C86C9B6E0BLUPR05MB306namprd_"
MIME-Version: 1.0
--_000_BLUPR05MB306826F771A3374E484C86C9B6E0BLUPR05MB306namprd_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Hello
I am contacting you for support regarding
https://stackoverflow.com/questions/46165415/configure-nginx-proxy-to-prese=
rve-the-chunk-size-sent-from-the-proxied-backend-s
While trying to use Nginx-Ingress-controller in K8s, ran into issue related=
to the behavior of nginx proxy,
The proxied backend server (our NGW) is set to return audio in chunks of ht=
tp to the client with multiparts & Some of our clients (already in the fiel=
d) have so far expected every single HTTP chunk to contain a full part of a=
n audio segment....ie a single audio segment may not be broken across sever=
al HTTP chunks.
For our use case, we had to set the proxy_buffering : off so to allow the =
nginx to send back results as soon as possible to not increase the CPL, But=
as a side effect, we noticed that the nginx was NOT trying to preserve the=
chunks sizes sent back by the proxied server (NGW), which would cause clie=
nts in the field to break.
So a single Audio part that is sent back by the NGW with a single http chun=
k of size 24838 may be broken into 5 or 6 chunks returned by nginx-proxy to=
the client with the total size amounting to the same thing.... We suspect=
this is due to the speed to read/write of the responses on Nginx, and it m=
ay read a single chunk in separate calls (example 12000 bytes then another =
11000 bytes then another 1838 bytes) and because buffering is OFF...these =
things are being sent back as independent chunks to the client.
We understand that the behavior of nginx is fully compliant with http chunk=
ing and that the client app should be better implemented to not assume coup=
ling of chunks and audio segments full parts, BUT, customers app already in=
the field cannot be helped,
Enabling proxy_buffering : on is not an option, as it delay responses until=
ALL the audio segments are ready (from all chunks) and it breaks the 1chun=
k to 1 audio segment rule as well.
Could you please advise if there is any way to control the nginx buffering =
to allow it to buffer per Chunk, ir do not send back received data unless a=
full Http chunk is read.... ?
Thank you for you support
Regards
Fares
--_000_BLUPR05MB306826F771A3374E484C86C9B6E0BLUPR05MB306namprd_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dus-ascii"=
>
<meta name=3D"Generator" content=3D"Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=3D"EN-US" link=3D"#0563C1" vlink=3D"#954F72">
<div class=3D"WordSection1">
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Hello<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">I am contacting you for support regarding <o:p></o:p=
></p>
<p class=3D"MsoNormal"><a href=3D"https://stackoverflow.com/questions/46165=
415/configure-nginx-proxy-to-preserve-the-chunk-size-sent-from-the-proxied-=
backend-s">https://stackoverflow.com/questions/46165415/configure-nginx-pro=
xy-to-preserve-the-chunk-size-sent-from-the-proxied-backend-s</a><o:p></o:p=
></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">While trying to use Nginx-Ingress-controller in K8s,=
ran into issue related to the behavior of nginx proxy,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">The proxied backend server (our NGW) is set to retur=
n audio in chunks of http to the client with multiparts & Some of our c=
lients (already in the field) have so far expected every single HTTP chunk =
to contain a full part of an audio segment….ie
a single audio segment may not be broken across several HTTP chunks.<o:p><=
/o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">For our use case, we had to set the <b><span s=
tyle=3D"color:red">proxy_buffering : off</span></b> so to allow the nginx t=
o send back results as soon as possible to not increase the CPL, But as a s=
ide effect, we noticed that the nginx was
NOT trying to preserve the chunks sizes sent back by the proxied server (N=
GW), which would cause clients in the field to break.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">So a single Audio part that is sent back by the NGW =
with a single http chunk of size 24838 may be broken into 5 or 6 chunks ret=
urned by nginx-proxy to the client with the total size amounting to t=
he same thing…. We suspect this is due
to the speed to read/write of the responses on Nginx, and it may read a si=
ngle chunk in separate calls (example 12000 bytes then another 11000 bytes =
then another 1838 bytes) and because buffering is OFF…these thi=
ngs are being sent back as independent chunks
to the client.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">We understand that the behavior of nginx is fully co=
mpliant with http chunking and that the client app should be better impleme=
nted to not assume coupling of chunks and audio segments full parts, BUT, c=
ustomers app already in the field
cannot be helped,<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Enabling <b><span style=3D"color:red">proxy_bufferin=
g : on</span></b> is not an option, as it delay responses until ALL the aud=
io segments are ready (from all chunks) and it breaks the 1chunk to 1 audio=
segment rule as well.<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Could you please advise if there is any way to contr=
ol the nginx buffering to allow it to buffer per Chunk, ir do not send back=
received data unless a full Http chunk is read…. ?<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Thank you for you support <o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Regards<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal">Fares<o:p></o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
<p class=3D"MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>
--_000_BLUPR05MB306826F771A3374E484C86C9B6E0BLUPR05MB306namprd_--
]