Need help running external program

I am running the line of code below from a shell script and it works fine,
however I am at a total loss on how i can run it from within a Python
script as every option I have tried fails and it appears to be down to the
escaping of certain characters.

Advertisements

Rigga wrote:
> Hi,
>
> I am running the line of code below from a shell script and it works fine,
> however I am at a total loss on how i can run it from within a Python
> script as every option I have tried fails and it appears to be down to the
> escaping of certain characters.
>
> wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
> 's/.*url="\([^"]*\)".*/\1/p'
If your problem is getting a python string without worrying about how to
escape the escape sequences, try:

Advertisements

Pink wrote:
> Rigga wrote:
>
>> Hi,
>>
>> I am running the line of code below from a shell script and it works
>> fine, however I am at a total loss on how i can run it from within a
>> Python script as every option I have tried fails and it appears to be
>> down to the escaping of certain characters.
>>
>> wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
>> 's/.*url="\([^"]*\)".*/\1/p'
> If your problem is getting a python string without worrying about how to
> escape the escape sequences, try:
>
> r"""wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
> 's/.*url="\([^"]*\)".*/\1/p'"""
>
> You should be able to pass this directly to a popen() function.

Hi,

Thanks for replying however I have just tried that and it does not seem to
work, it doesnt return any results (i take it the r was a typo)

Rigga wrote:
> Pink wrote:
>
>> Rigga wrote:
>>
>>> Hi,
>>>
>>> I am running the line of code below from a shell script and it works
>>> fine, however I am at a total loss on how i can run it from within a
>>> Python script as every option I have tried fails and it appears to be
>>> down to the escaping of certain characters.
>>>
>>> wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
>>> 's/.*url="\([^"]*\)".*/\1/p'
>> If your problem is getting a python string without worrying about how to
>> escape the escape sequences, try:
>>
>> r"""wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
>> 's/.*url="\([^"]*\)".*/\1/p'"""
>>
>> You should be able to pass this directly to a popen() function.
>
> Hi,
>
> Thanks for replying however I have just tried that and it does not seem to
> work, it doesnt return any results (i take it the r was a typo)
>
> Thanks
>
> RiGGa

No, the r was the point - it's there to tell Python not to do any escaping
on the string. Try it again with the r and see what happens.

Tim Jarman wrote:
> Rigga wrote:
>
>> Pink wrote:
>>
>>> Rigga wrote:
>>>
>>>> Hi,
>>>>
>>>> I am running the line of code below from a shell script and it works
>>>> fine, however I am at a total loss on how i can run it from within a
>>>> Python script as every option I have tried fails and it appears to be
>>>> down to the escaping of certain characters.
>>>>
>>>> wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
>>>> 's/.*url="\([^"]*\)".*/\1/p'
>>> If your problem is getting a python string without worrying about how to
>>> escape the escape sequences, try:
>>>
>>> r"""wget -q www.anywebpage.com -O - | tr '\r' '\n' | tr \' \" | sed -n
>>> 's/.*url="\([^"]*\)".*/\1/p'"""
>>>
>>> You should be able to pass this directly to a popen() function.
>>
>> Hi,
>>
>> Thanks for replying however I have just tried that and it does not seem
>> to work, it doesnt return any results (i take it the r was a typo)
>>
>> Thanks
>>
>> RiGGa
>
> No, the r was the point - it's there to tell Python not to do any escaping
> on the string. Try it again with the r and see what happens.
>
Brilliant!!! that works a treat thankyou!!, where on earth did you find out
about the 'r' any pointers to documentation appreciated.

<SNIP>
>>No, the r was the point - it's there to tell Python not to do any escaping
>>on the string. Try it again with the r and see what happens.
>>
>
> Brilliant!!! that works a treat thankyou!!, where on earth did you find out
> about the 'r' any pointers to documentation appreciated.
>
> Thanks
>
> RiGGa

Rigga wrote:
>> No, the r was the point - it's there to tell Python not to do any
>> escaping on the string. Try it again with the r and see what happens.
>>
> Brilliant!!! that works a treat thankyou!!, where on earth did you find
> out
> about the 'r' any pointers to documentation appreciated.
This is a pretty common problem when working with regular expression (which
usually contain many backslashes) - that's where I saw this syntax for the
first time (e.g. http://docs.python.org/lib/match-objects.html).
The official reference for string literals is here:http://docs.python.org/ref/strings.html

Brian van den Broek wrote:
> Rigga said unto the world upon 2005-02-27 15:04:
>> Tim Jarman wrote:
>
> <SNIP>
>
>>>No, the r was the point - it's there to tell Python not to do any
>>>escaping on the string. Try it again with the r and see what happens.
>>>
>>
>> Brilliant!!! that works a treat thankyou!!, where on earth did you find
>> out
>> about the 'r' any pointers to documentation appreciated.
>>
>> Thanks
>>
>> RiGGa
>
> <http://www.python.org/doc/current/ref/strings.html>
> <http://www.python.org/doc/current/lib/module-re.html>
>
> Best,
>
> Brian vdB
Thanks for all your help with this it is appreciated, one further question
though, how do I pass a variable to the external program while using the
r"""

(snip stuff about raw strings)
> Thanks for all your help with this it is appreciated, one further question
> though, how do I pass a variable to the external program while using the
> r"""
>
> Thanks
>
> RiGGa

I'm not sure I understand the question. Say you have:

parameter = r"my \funky \text"

then surely you just pass it to your external program using whichever method
you like, e.g.

But it does not work, if I escape the string using r""" and hard code in the
web address rather than use %s and feed[counter] it works, my question is
how do I escape the string to get it to work with the %s and feed[counter]

(snip)
>>
> This is the command I am trying to run:
>
> feed is a list of web addresses
>
> output, input = popen2("wget -q %s -O - | tr '\r' '\n' | tr \' \" | sed -n
> 's/.*url="\([^"]*\)".*/\1/p'" % feed[counter])
>
> But it does not work, if I escape the string using r""" and hard code in
> the web address rather than use %s and feed[counter] it works, my question
> is how do I escape the string to get it to work with the %s and
> feed[counter]
>
> Im new to python as you can tell

Disclaimer: I know nothing about wget beyond what just having typed 'man
wget' told me!

1. What *exactly* does "it does not work" mean? Do you get a traceback? If
so, post it - that will help others to help you. What results are you
expecting?

2. Are you sure feed contains what you think it contains at this point in
your program? What do you see if you do:

for thing in feed: print thing

?

A good strategy in these cases is to go in small steps. before you try
getting fancy with popen2, have your program just print the command-line
correctly. Then maybe try it with something like echo just to see that
you're passing what you think you're passing. And so on.

Rigga wrote:
> Tim Jarman wrote:
>
>
>>Rigga wrote:
>>
>>
>>>Brian van den Broek wrote:
>>>
>>>
>>>>Rigga said unto the world upon 2005-02-27 15:04:
>>
>>(snip stuff about raw strings)
>>
>>
>>>Thanks for all your help with this it is appreciated, one further
>>>question though, how do I pass a variable to the external program while
>>>using the r"""
>>>
>>>Thanks
>>>
>>>RiGGa
>>
>>I'm not sure I understand the question. Say you have:
>>
>>parameter = r"my \funky \text"
>>
>>then surely you just pass it to your external program using whichever
>>method you like, e.g.
>>
>>import os
>>os.execl("your_external_prog", parameter) # replaces the current process
>>
>>or some variant of:
>>
>>return_code = os.spawnl(os.P_WAIT, "your_external_prog", parameter)
>>
>>or you can build a command line:
>>
>>command = "your_external_prog %s" % parameter
>>return_code = os.system(command)
>>
>>(see docs on the os module for more variations on this theme than you can
>>shack a stick at)
>>
>>It's just a string, after all.
>>
>>
>>
>
> This is the command I am trying to run:
>
> feed is a list of web addresses
>
> output, input = popen2("wget -q %s -O - | tr '\r' '\n' | tr \' \" | sed -n
> 's/.*url="\([^"]*\)".*/\1/p'" % feed[counter])
>
> But it does not work, if I escape the string using r""" and hard code in the
> web address rather than use %s and feed[counter] it works, my question is
> how do I escape the string to get it to work with the %s and feed[counter]
>
> Im new to python as you can tell

Right, using raw strings (r" ... ") makes sure that backslashes in the
literal are retained rather than used as escapes. Socould you show us an
example where the expression (using ... % feed[counter]) gives you a
different value from "hard coding the web address"?

I suspect if you use a raw string for the format then that will be
enough - in other words, does

Thanks to all for your help it is now working, I rant he code through a
debugger and found that the input file I was using to create my list of
addresses to wget had newlines in them and were therefore breaking my
command line.

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!