Re: [Webware-discuss] The appserver crash when high demand?

Marcelo Ametller wrote:
> yes: from cStringIO import StringIO
> I suppose that use the memory of the servidor?
Yes, it provides a kind of memory file for Reportlab to write to instead
a real file on the harddisk. In this case, it would not make much
difference, but if you serve many small PDFs it is better. You also
don't have to care about a unique temporary file name. Sometimes it may
also make sense to cache the PDFs you created. I.e. you don't create a
temporary filename, but a filename that is made from the input
parameters (e.g. weekly_report_03_2006). If somebody else requests the
same document, you can serve it directly from disk.
-- Christoph

Thread view

hello
my plataform:
ubuntu breezy, apache2, webware0.9 , webkit, webform, postgres8 , python2.4
my hard:
celeron 1.8 ram 384Mb hd 40gb
my problem!
I have a servlet, with a form, a button with label "generate".
When the user press the button, de "action" run a function, it generate
a pdf (reportlab).
When the pdf have 200 items, that's ok. But when the pdf have 6000
items, the appserver crash. The dmesg inform that there isn't memory...
However if I not use webware,webkit, the python script works great with
6000 items.
The same sussed when i process very data.
Does anybody knows what's sussed?
Thank you all.
Marcelo

marcelo ametller wrote:
> I have a servlet, with a form, a button with label "generate".
> When the user press the button, de "action" run a function, it generate
> a pdf (reportlab).
> When the pdf have 200 items, that's ok. But when the pdf have 6000
> items, the appserver crash. The dmesg inform that there isn't memory...
Does this crash immediately after you start the appserver and generate
the very first pdf? Or does it only happen after some time when more
users have been on the server?
What do you get in the Webware logfile?
Do you create the pdf in memory only or do you write it to a file?
-- Christoph

On 1/21/06, Christoph Zwerschke <cito@...> wrote:
> marcelo ametller wrote:
> > I have a servlet, with a form, a button with label "generate".
> > When the user press the button, de "action" run a function, it generate
> > a pdf (reportlab).
> > When the pdf have 200 items, that's ok. But when the pdf have 6000
> > items, the appserver crash. The dmesg inform that there isn't memory...
>
> Does this crash immediately after you start the appserver and generate
> the very first pdf? Or does it only happen after some time when more
> users have been on the server?
>
> What do you get in the Webware logfile?
>
> Do you create the pdf in memory only or do you write it to a file?
>
> -- Christoph
Also, can do you this generation outside of Webware, say as a
standalone Python program? In the past, most, if not all, crashes have
been associated with compiled Python extensions.
-Chuck

Christoph Zwerschke wrote:
> marcelo ametller wrote:
>
>> I have a servlet, with a form, a button with label "generate".
>> When the user press the button, de "action" run a function, it generate
>> a pdf (reportlab).
>> When the pdf have 200 items, that's ok. But when the pdf have 6000
>> items, the appserver crash. The dmesg inform that there isn't memory...
>
>
> Does this crash immediately after you start the appserver and generate
> the very first pdf? Or does it only happen after some time when more
> users have been on the server?
it only crash after some time... the time for generate de pdf is 20min.
>
> What do you get in the Webware logfile?
nothing...
>
> Do you create the pdf in memory only or do you write it to a file?
i not know that...
i paste de code here:
action for the botton:
def generar(self):
fecha1=self.form.field('fecha1').value()
fecha2=self.form.field('fecha2').value()
self.session().setValue('fecha1', fecha1)
self.session().setValue('fecha2', fecha2)
url='/menu/fac/inf/v/libFactR1'
self.response().sendRedirect(url)
self.writeBody()
part of the servlet:
def writeHTML(self):
req=self.request()
self.fecha1=self.session().value('fecha1')
self.fecha2=self.session().value('fecha2')
self.empresa = empresa.nombre
self.titulo = "Resumen Libro Facturacion - Facturas generadas entre %s
- %s" % (self.fecha1,self.fecha2)
fact= Fact1()
#consulta sql (return 6000 regs for 10 columns)
self.listado= fact.lisLibFactR1(self.fecha1,self.fecha2)
buffer=StringIO()
doc = SimpleDocTemplate(buffer,pagesize=A4,leftMargin=3*cm)
doc.setTitle=self.titulo
doc.setAuthor='operador'
Story = []
data=self.listado
colwidths=(10*cm,2*cm)
t=Table( data, colwidths)
t.setStyle(TableStyle([('VALIGN',(0,0),(-1,-1),'MIDDLE'),
('GRID',(0,0),(-1,-1), 0,colors.grey), ('BOX', (0,0), (-1,-1),
1, colors.black), ]))
Story.append(t)
Story.append(Spacer(1,0.2*cm))
doc.build(Story, onFirstPage=self.myFirstPage,
onLaterPages=self.myLaterPages)
pdf=buffer.getvalue()
buffer.close()
self.response().setHeader('Content-Type', 'application/pdf')
self.response().setHeader('Content-Length', str(len(pdf)))
self.response().setHeader('Content-Disposition', 'inline;
filename="ReslibFact.pdf"')
self.write(pdf)
thanks for you time...
Marcelo

marcelo ametller wrote:
> it only crash after some time... the time for generate de pdf is 20min.
That's way too long. If it really must take so long, you should spawn a
separate process to create the pdf. Your webserver will probably quit
the connection with timeout anyway.
>> What do you get in the Webware logfile?
> nothing...
You can insert print statements, and use unbuffered output for the
AppServer, to see where the servlet is spinning. But after a look on
your servlet I am almost sure it will be the doc.build() call.
>> Do you create the pdf in memory only or do you write it to a file?
> i not know that...
> i paste de code here:
That would be in memory. Can you check whether the StringIO is imported
from cStringIO (not StringIO)?
But the real problem is probably here:
> t=Table( data, colwidths)
You should definitely use "LongTable" instead of "Table" if your table
has so many rows. The time used by Table goes quadratic with the number
of rows...
The rest looks fine.
Let us know whether that helps.
-- Christoph

Christoph Zwerschke wrote:
> marcelo ametller wrote:
>
>> it only crash after some time... the time for generate de pdf is 20min.
>
>
> That's way too long. If it really must take so long, you should spawn a
> separate process to create the pdf. Your webserver will probably quit
> the connection with timeout anyway.
>
>>> What do you get in the Webware logfile?
>>
>> nothing...
>
>
> You can insert print statements, and use unbuffered output for the
> AppServer, to see where the servlet is spinning. But after a look on
> your servlet I am almost sure it will be the doc.build() call.
>
>>> Do you create the pdf in memory only or do you write it to a file?
>>
>> i not know that...
>> i paste de code here:
>
>
> That would be in memory. Can you check whether the StringIO is imported
> from cStringIO (not StringIO)?
yes: from cStringIO import StringIO
I suppose that use the memory of the servidor?
>
> But the real problem is probably here:
>
>> t=Table( data, colwidths)
now use:
from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer,
LongTable, TableStyle
the time of that form is smaller...
>
>
> You should definitely use "LongTable" instead of "Table" if your table
> has so many rows. The time used by Table goes quadratic with the number
> of rows...
>
> The rest looks fine.
>
> Let us know whether that helps.
it's the appserver inf:
empezo doc.build
13 2006-01-24 09:51:39 /menu/fac/inf/v/auxIva1
termino doc.build
13 234.81 secs /menu/fac/inf/v/auxIva
the pdf contain 180 pages.
the appserver not crash...
thanks...
--
Marcelo

Marcelo Ametller wrote:
> yes: from cStringIO import StringIO
> I suppose that use the memory of the servidor?
Yes, it provides a kind of memory file for Reportlab to write to instead
a real file on the harddisk. In this case, it would not make much
difference, but if you serve many small PDFs it is better. You also
don't have to care about a unique temporary file name. Sometimes it may
also make sense to cache the PDFs you created. I.e. you don't create a
temporary filename, but a filename that is made from the input
parameters (e.g. weekly_report_03_2006). If somebody else requests the
same document, you can serve it directly from disk.
-- Christoph

Community

Help

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

I agree to receive quotes, newsletters and other information from sourceforge.net and its partners regarding IT services and products. I understand that I can withdraw my consent at any time. Please refer to our Privacy Policy or Contact Us for more details