best way to replace first word in string?

Guest

I am looking for the best and efficient way to replace the first word
in a str, like this:
"aa to become" -> "/aa/ to become"
I know I can use spilt and than join them
but I can also use regular expressions
and I sure there is a lot ways, but I need realy efficient one

I'm sure the regular expression gurus here can come
up with something if it can be followed by other than
a space.

-Larry Bates

wrote:
> I am looking for the best and efficient way to replace the first word
> in a str, like this:
> "aa to become" -> "/aa/ to become"
> I know I can use spilt and than join them
> but I can also use regular expressions
> and I sure there is a lot ways, but I need realy efficient one
>

Advertisements

"" <> writes:
> I am looking for the best and efficient way to replace the first word
> in a str, like this:
> "aa to become" -> "/aa/ to become"
> I know I can use spilt and than join them
> but I can also use regular expressions
> and I sure there is a lot ways, but I need realy efficient one

Assuming you know the whitespace will be spaces, I like find:

new = "/aa/" + old[old.find(' '):]

As for efficiency - I suggest you investigate the timeit module, and
do some tests on data representative of what you're actaully going to
be using.

On Oct 20, wrote:
> I am looking for the best and efficient way to replace the first word
> in a str, like this:
> "aa to become" -> "/aa/ to become"
> I know I can use spilt and than join them
> but I can also use regular expressions
> and I sure there is a lot ways, but I need realy efficient one

Of course there are many ways to skin this cat; here are some trials.
The timeit module is useful for comparison (and I think I'm using it
correctly . I thought that string concatenation was rather
expensive, so its being faster than %-formatting surprised me a bit:

the above has to look the pattern up in the compilation cache for each loop,
and it also has to parse the template string. precompiling the pattern and
using a callback instead of a template string can speed things up somewhat:

On Thu, 20 Oct 2005 08:26:43 -0700, wrote:
> I am looking for the best and efficient way to replace the first word
> in a str, like this:
> "aa to become" -> "/aa/ to become"
> I know I can use spilt and than join them
> but I can also use regular expressions
> and I sure there is a lot ways, but I need realy efficient one

Efficient for what?

Efficient in disk-space used ("least source code")?

Efficient in RAM used ("smallest objects and compiled code")?

Efficient in execution time ("fastest")?

Efficient in program time ("quickest to write and debug")?

If you use regular expressions, does the time taken in loading the module
count, or can we assume you have already loaded it?

It will also help if you specify your problem a little better. Are you
replacing one word, and then you are done? Or at you repeating hundreds of
millions of times? What is the context of the problem?

Most importantly, have you actually tested your code to see if it is
efficient enough, or are you just wasting your -- and our -- time with
premature optimization?

<> wrote:
> I am looking for the best and efficient way to replace the first word
> in a str, like this:
> "aa to become" -> "/aa/ to become"
> I know I can use spilt and than join them
> but I can also use regular expressions
> and I sure there is a lot ways, but I need realy efficient one

(This should not be thought of as the exact process that Python uses, but
simply illustrating the general procedure.)

Now think of what str-formatting would do:

s = "hello %s" % "world"

In pseudo-code, it might do something like this:

- Allocate a chunk of bytes, hopefully not too big or too small.
- Repeat until done:
- Copy chars from the original string into the new string,
until it hits a %s placeholder.
- Grab the next string from the args, and copy chars from
that into the new string. If the new string is too small,
reallocate memory to make it bigger, potentially moving
chunks of bytes around.

The string formatting pseudo-code is a lot more complicated and has to do
more work than just blindly copying bytes. It has to analyse the bytes it
is copying, looking for placeholders.

So string concatenation is more efficient, right? No. The thing is, a
*single* string concatenation is almost certainly more efficient than a
single string concatenation. But now look what happens when you repeat it:

On 2005-10-22, William Park wrote:
> <> wrote:
>> I am looking for the best and efficient way to replace the first word
>> in a str, like this:
>> "aa to become" -> "/aa/ to become"
>> I know I can use spilt and than join them
>> but I can also use regular expressions
>> and I sure there is a lot ways, but I need realy efficient one
>
> I doubt you'll find faster than Sed.

On the contrary; to change a string, almost anything will be faster
than sed (except another external program).

If you are in a POSIX shell, parameter expansion will be a lot
faster.

In a python program, one of the solutions already posted will be
much faster.

Chris F.A. Johnson <> wrote:
> On 2005-10-22, William Park wrote:
> > <> wrote:
> >> I am looking for the best and efficient way to replace the first word
> >> in a str, like this:
> >> "aa to become" -> "/aa/ to become"
> >> I know I can use spilt and than join them
> >> but I can also use regular expressions
> >> and I sure there is a lot ways, but I need realy efficient one
> >
> > I doubt you'll find faster than Sed.
>
> On the contrary; to change a string, almost anything will be faster
> than sed (except another external program).
>
> If you are in a POSIX shell, parameter expansion will be a lot
> faster.
>
> In a python program, one of the solutions already posted will be
> much faster.

On Sat, 22 Oct 2005 21:41:58 +0000, Ron Adam wrote:
> Don't forget a string can be sliced. In this case testing before you
> leap is a win. ;-)

Not much of a win: only a factor of two, and unlikely to hold in all
cases. Imagine trying it on *really long* strings with the first space
close to the far end: the split-and-join algorithm has to walk the string
once, while your test-then-index algorithm has to walk it twice.

So for a mere factor of two benefit on short strings, I'd vote for the
less complex split-and-join version, although it is just a matter of
personal preference.

On 2005-10-22, William Park wrote:
> Chris F.A. Johnson <> wrote:
>> On 2005-10-22, William Park wrote:
>> > <> wrote:
>> >> I am looking for the best and efficient way to replace the first word
>> >> in a str, like this:
>> >> "aa to become" -> "/aa/ to become"
>> >> I know I can use spilt and than join them
>> >> but I can also use regular expressions
>> >> and I sure there is a lot ways, but I need realy efficient one
>> >
>> > I doubt you'll find faster than Sed.
>>
>> On the contrary; to change a string, almost anything will be faster
>> than sed (except another external program).
>>
>> If you are in a POSIX shell, parameter expansion will be a lot
>> faster.
>>
>> In a python program, one of the solutions already posted will be
>> much faster.
>
> Care to put a wager on your claim?

In a shell, certainly.

If one of the python solutions is not faster than sed (e.g.,
os.system("sed .....")) I'll forget all about using python.

On Sat, 22 Oct 2005 14:54:24 -0400, Mike Meyer wrote:
>> The string formatting is two orders of magnitude faster than the
>> concatenation. The speed difference becomes even more obvious when you
>> increase the number of strings being concatenated:
>
> The test isn't right - the addition test case includes the time to
> convert the number into a char, including taking a modulo.

I wondered if anyone would pick up on that

You are correct, however that only adds a constant amount of time to
the time it takes for each concatenation. That's why I talked about order
of magnitude differences. If you look at the vast increase in time taken
for concatenation when going from 10**5 to 10**6 iterations, that cannot
be blamed on the char conversion.

At least, that's what it looks like to me -- I'm perplexed by the *vast*
increase in speed in your version, far more than I would have predicted
from pulling out the char conversion. I can think of three
possibilities:

(1) Your PC is *hugely* faster than mine;

(2) Your value of x is a lot smaller than I was using (you don't actually
say what x you use); or

(3) You are using a version and/or implementation of Python that has a
different underlying implementation of string concatenation.

> I couldn't resist adding the .join idiom to this test:

[snip code]
>>>> tester(x)
> 0.0551731586456 0.0251281261444 0.0264830589294
>>>> tester(x)
> 0.0585241317749 0.0239250659943 0.0256059169769
>>>> tester(x)
> 0.0544500350952 0.0271301269531 0.0232360363007
>
> The "order of magnitude" now falls to a factor of two. The original
> version of the test on my box also showed an order of magnitude
> difference, so this isn't an implementation difference.

Looking just at the improved test of string concatenation, I get times
about 0.02 second for n=10**4. For n=10**5, the time blows out to 2
seconds. For 10**6, it explodes through the roof to about 2800 seconds, or
about 45 minutes, and for 10**7 I'm predicting it would take something of
the order of 500 HOURS.

In other words, yes the char conversion adds some time to the process, but
for large numbers of iterations, it gets swamped by the time taken
repeatedly copying chars over and over again.

Steven D'Aprano wrote:
> On Sat, 22 Oct 2005 21:41:58 +0000, Ron Adam wrote:
>
>
>>Don't forget a string can be sliced. In this case testing before you
>>leap is a win. ;-)
>
>
> Not much of a win: only a factor of two, and unlikely to hold in all
> cases. Imagine trying it on *really long* strings with the first space
> close to the far end: the split-and-join algorithm has to walk the string
> once, while your test-then-index algorithm has to walk it twice.
>
> So for a mere factor of two benefit on short strings, I'd vote for the
> less complex split-and-join version, although it is just a matter of
> personal preference.
>

Guess again... Is this the results below what you were expecting?

Notice the join adds a space to the end if the source string is a single
word. But I allowed for that by adding one in the same case for the
index method.

The big win I was talking about was when no spaces are in the string.
The index can then just return the replacement.

These are relative percentages of time to each other. Smaller is better.

Steven D'Aprano <> writes:
> On Sat, 22 Oct 2005 14:54:24 -0400, Mike Meyer wrote:
>>> The string formatting is two orders of magnitude faster than the
>>> concatenation. The speed difference becomes even more obvious when you
>>> increase the number of strings being concatenated:
>> The test isn't right - the addition test case includes the time to
>> convert the number into a char, including taking a modulo.
> I wondered if anyone would pick up on that
> You are correct, however that only adds a constant amount of time to
> the time it takes for each concatenation. That's why I talked about order
> of magnitude differences. If you look at the vast increase in time taken
> for concatenation when going from 10**5 to 10**6 iterations, that cannot
> be blamed on the char conversion.

True. string addition is O(n^2); the conversion time is O(n). But
fair's fair.
> At least, that's what it looks like to me -- I'm perplexed by the *vast*
> increase in speed in your version, far more than I would have predicted
> from pulling out the char conversion. I can think of three
> possibilities:

Everything got faster, so it wasn't just pulling the chr conversion.
> (1) Your PC is *hugely* faster than mine;

It's a 3Ghz P4.
> (2) Your value of x is a lot smaller than I was using (you don't actually
> say what x you use); or

It's still in the buffer, and I copied it from your timings:
>>> x = 100000
> (3) You are using a version and/or implementation of Python that has a
> different underlying implementation of string concatenation.

Share This Page

Welcome to The Coding Forums!

Welcome to the Coding Forums, the place to chat about anything related to programming and coding languages.

Please join our friendly community by clicking the button below - it only takes a few seconds and is totally free. You'll be able to ask questions about coding or chat with the community and help others.
Sign up now!