sh: neither as fast as sloths nor as elegant as hagfish

[fivemack]

I have a directory with 244 files with names like m12331246123468911531238951802368109467.mlog, which I want to rename to names like C038.123312.mlog

time for u in m*mlog; do B=$(echo $u | cut -dm -f2 | cut -d. -f1); echo $u C${#B}.$(echo $B | cut -c1-6).mlog; done

takes 17 seconds

time for u in m*mlog; do B=$(echo $u | cut -dm -f2 | cut -d. -f1); echo $u C${#B}.${B:0:6}.mlog; done

takes eight seconds

time for u in m*mlog; do B=${u:1}; B=${B%.mlog}; echo $u C${#B}.${B:0:6}.mlog; done

takes 0.2 seconds.

Of course, when I replace 'echo' with 'mv' it still takes fourteen seconds, but I am not that shocked that mv over NFS might be slow.

Which suggests that doing $() to start a new shell is taking something like a hundredth of a second on a one-year-old PC. I didn't know that. On the other hand, if I start writing code this dense in unclear bashisms, my colleagues at work will disembowel me with spoons.

PS: if I stop running a CPU-intensive program on each of my eight cores, starting new processes gets about fifteen times faster. I can understand if it got twice as fast, but I really don't understand fifteen.

Comments

[fivemack.livejournal.com]

The machine has eight CPUs, so I would have assumed my new bash has to take its place in one of the eight per-CPU round-robins, which would cause it to go half as fast as were there no heavy-CPU jobs running.

[dd-b.livejournal.com]

Hmmm; the Linux 2.6 scheduler does seem to have per-CPU run queues. That surprises me, since generally single-queue multi-server is much fairer and more efficient at allocating fungible resources. However, possibly the benefits of CPU affinity (mostly hot cache contents) trump that. And it's preemptive. So your task should in fact hit a cpu immediately (bumping the long-running task, whose priority will have been gradually raised (higher number, lower precedence)).

So yeah, there's something to explain there.

You're not short of memory for what's running, are you?

[pjc50.livejournal.com]

While it is preemptive, I don't think it will evict a running task from a cpu in favour of a new one. Moreover the shell script is going to repeatedly lose its scheduler slot due to waiting for a response from the NFS server.