Fog Creek Software
Discussion Board




Linux Commands

I've been trying to brush up lately on my Linux skills and I constantly find myself annoyed with all of the inconsistancies between commands.

In particular, I've been looking at sed and awk and tcl/tk and I'm wondering "can't I do all of this with perl and dispense with learning a 5 or 10 different tools?" It seems like if I could devote my time to learning perl really really well, I could be much more effective than if I learn all of the other loosely-related tools.

Of course, I'm only vaguely familiar with any of these toos, and there is very likely something that I'm missing. Fill me in. Enlighten me.

Benji Smith
Wednesday, November 06, 2002

I think that's what perl is for.  So yes.

Acowymous Nonerd
Wednesday, November 06, 2002

Part of the "fun" is learning all those commands!

And remember that Unix philisophy is small, single-purpose tools strung together with pipes or bundled into scripts. It's just the name of the game.

Spiny Norman
Wednesday, November 06, 2002

Stick with it, it'll come. Perl's great, but not always the right thing to use. If you're going to get along in the *nix world, you're pretty much (IMO, anyway) going to have to get used to the variety of basic tools such as the ones you mentioned.

anonQAguy
Wednesday, November 06, 2002

By the way, I should probably clarify:

I'm really not interested in doing much sysadmin type work. What I'm really interested in is running scientific simulations, parsing text for NLP (Natural Language Processing) algorithms, creating the occasional web application, and that type of stuff.

I'm sure I will, from time to time, have to do some sysadmin stuff (because you really can't avoid it), but I'd rather do that stuff with perl, If I can.

I'm really not interested in the "fun" of learning a bazillion different commands, each with their own syntax. And I really don't care what the Unix philosophy is. I just want to make the most efficient use of _my_ time as I learn all of this stuff.

Benji Smith
Wednesday, November 06, 2002

...then Perl sounds about as right as it gets for you..

Eric DeBois
Wednesday, November 06, 2002

Why learn sed and awk if you know perl?

perl -ne 's/foo/bar/g;print;'

there is also a few flags for 'perform this operation on the file given' so it replaces them within the file, and a few other useful flags too.  I've been using linux for about 4 years now, and perl for 3, and occasionally I'll use sed or awk for fun, but if I need to get it done I use perl.

Though, one book has come in really handy for me, its worth its weight in gold, for unix users:
http://www.oreilly.com/catalog/upt3/
Its a great book.  And I see the 3rd edition is out!  Time to get out the checkbook...

So I guess in summary, you'll pick up the commands as you need to learn them, and the best thing you can know is where to look for more information.  Also, save all of your complex / handy commands in a scripts directory, and document them.  They will come in handy later.

Andrew Hurst
Wednesday, November 06, 2002

I have not used perl much.
I use php for web stuff.
http://www.php.net/manual/en/
has a very smart dynamic manual, that lets user add thing like advanced tips etc.

I sometimes use sed and awk, and this is a page I often come back to when I need to brush up on sed fast :
http://www-h.eng.cam.ac.uk/help/tpl/unix/sed.html

grep, sort, wc, uniq are some other small commands I use in scripts.

Fred
Wednesday, November 06, 2002

Remember,  sed, awk, tcl/tk, and all the other programs have been developed over time since the early days of Unix (they aren't "Linux Commands"). Most were written to 'scratch an itch' and other people have found them useful too.

If Perl does everything you need, don't bother learning them. They're all just tools - use the one that lets you get your job done.

jeff
Wednesday, November 06, 2002

Agreed with the posts above; dive into Perl, and do as much as you can there.  Then try other tools if you feel that you need them.

OTOH, remember that other tools can be useful in their own ways.  Lex and yacc can be strung together to create a simple compiler, just in and of themselves.

Brent P. Newhall
Wednesday, November 06, 2002

I recently replaced a combination of shell, sed and awk scripts with Python and I couldn't be happier.  The same would probably apply for Perl.  For my money, the benefit comes from having a common syntax. 
So go ahead and forget sed and awk.  If you find you still need some information on them, get the Oreilly pocket book.  It also comes in handy as a regular expression guide.

Scott Rogers
Wednesday, November 06, 2002

I like perl, but it's a mistake to clump TCL/TK in with sed and awk - it's a far more complete programming language, and GUI work is very simple with it.

I've been doing a lot of TCL work in the past two years, and I can do pretty much anything in TCL that I can do in Perl (and it's more trivially extensible).  The main drawback is the lack of extensions compared to the monster that is CPAN.

Of course, if you like Perl and are happy with it, messing about with TCL is probably a waste of time.

As far as other shell programming goes, it's mostly useful on the system admin/system integrator level.

Rodger Donaldson
Wednesday, November 06, 2002

Though it is not uncommon to find perl installed on systems now, that has not always the case. I have worked on several flavors of unix over the years and sed and awk have always been there.
Similarly if you need portability across multiple unix platforms, /bin/sh is always there when you need a consistent scripting framework.

DB
Wednesday, November 06, 2002

I would also consider ruby as an alternative to perl. I tried learning perl myself, by writing a unit testing utility for c/c++ in it. It took a week of extremely hard work and had limited functionality. The first time I was exposed to ruby, I rewrote it in ruby as an exercise. It took an afternoon, was clearer and easier to understand. I'm not claiming that ruby is inherently superior to perl, just that as an experienced OO programmer, I found it much easier to get my head round, and seemed to have the same functionality.

regards, steve

stephen hill
Thursday, November 07, 2002

-- DB said --
Similarly if you need portability across multiple unix platforms, /bin/sh is always there when you need a consistent scripting framework.
-------

Not necessarily.  There are some linux distributions that replace it with bash, or some flavor of bash.  I seem to recall some problems it caused, but I can't remember them offhand.  I think you have to run bash in sh emulation mode somehow (maybe it can tell by the /bin/sh line at the top... I dunno).

Anyway, I need more coffee, and my point is that that statement isn't always correct.

Andrew Hurst
Thursday, November 07, 2002

The big problem with /bin/sh scripts is not that sh isn't always there. The problem is that you can't do ANYTHING in a shell script without calling external programs. Only if all of the external programs are on the system, and have the exact same list of parameters, will your script really be portable.

And there are no two unix systems that have the same set of parameters. Just try and get a directory listing with attributes and sorted by size on Linux, HP/UX, and Solaris, and try to do it portably, without cheating by installing something like the gnu version of ls.

Shell scripts suck.

-Chris, who's dealt with porting way too many shell scripts lately.

Chris Tavares
Thursday, November 07, 2002

I suppose the only always is that there are always exceptions to the rule.

My biggest scripting headaches have been when I have needed to take a script written with ksh or bash in mind and having to fix it to work properly with a system that does not have one or the other. 

/bin/sh (Bourne shell to be exact)  is the lowest common denominator that I have found and coding to that makes my life easier.  Thats not to say that I do not have to use some uname based cases in my scripts. There are always exceptions. :-)

DB
Thursday, November 07, 2002

"And there are no two unix systems that have the same set of parameters. Just try and get a directory listing with attributes and sorted by size on Linux, HP/UX, and Solaris, and try to do it portably, without cheating by installing something like the gnu version of ls."

Cheating?  Isn't that just being smart?

I never understand why people try to do things like write portable makefiles.  Just use GNU make and be done with it.  All the portability you need is already in GNU make.

Bob

Robert Anderson
Friday, November 08, 2002

"What I'm really interested in is running scientific simulations"

Benji:

You may want to consider using Scilab, which is a MatLab like high level language. Octave is another MatLab like software.  Finally there is GNU Scientific Libraries (GSL).
http://www-rocq.inria.fr/scilab/
http://www.octave.org/
http://sources.redhat.com/gsl/

Scilab will also run on Windows 95/98/2000

Joe
Friday, November 08, 2002

>>> I never understand why people try to do things like write portable makefiles.  Just use GNU make and be done with it.  All the portability you need is already in GNU make.
<<<

Well, because GNU make itself needs to be compiled?

Actually, I'm being serious. I have a project that needs to run on both Win32 and HP/UX 11i. GNU make does NOT compile out of the box on HP/UX. It took me quite a bit of searching to find a binary.

Chris Tavares
Friday, November 08, 2002

"Well, because GNU make itself needs to be compiled?  Actually, I'm being serious. I have a project that needs to run on both Win32 and HP/UX 11i. GNU make does NOT compile out of the box on HP/UX. It took me quite a bit of searching to find a binary."

It should.  Contact the maintainer Paul D. Smith - he's quite responsive and knowledgeable.  A "port" is probably a trivial extension of the build flags.

"Factoring up" the portability into GNU make rather than trying to write portable makefiles still seems like a no-brainer to me.

Bob

Robert Anderson
Friday, November 08, 2002

Okay, I just found out that a TRUE boolean value is represented as a ZERO when working with shell scripts. NON-ZERO values evaluate to FALSE.

That's just plain stupid.

Benji Smith
Wednesday, November 13, 2002

*  Recent Topics

*  Fog Creek Home