|
When you have to be really quick
Dear Joel and participants of the forum,
I wonder why all your articles are about more or less big software projects
developing some end-user applications from scratch. There are many other
sides of programming. For me, the most important side usually is
creating simple tools for solving a specific problem, where the development
time MUST not be more than a few minutes. I'd like to have your opinion -
and the opinion of other people here in the forum - about an example
problem I describe on what programming language would you use.
Situation: you are a consultant of an accounting software. You import
data from an other software. You find out that the file format is not
good, if column 8 is "foo" then column 3 must be "bar".
If you wonder, why would such thing happen, I will
explain later. If you wonder, why don't just do it in Excel, I will
also explain that later. So, you have to write a quick program to
do it.
Now, the problem is, that you don't have time to start typing "public
static void"... I will also explain that later why. You want to
do it really, really fast. What language would you use?
My solutions:
I'd use CCC. CCC stands for Clipper to C++ Compiler, and is developed
by a Hungarian company with with a way of thinking "do things really fast".
a) run FAR Manager (www.rarlab.com)
b) Shift-F4, type fuckthiscsv.prg
c)
function main(fn)
local in,line,out
in:=fopen(fn)
out:=fcreate(fname(fn)+"2"+fext(fn))
do while (NIL!=(line:=freadcsvln(in)))
if (line[8]=="foo")
line[3]:="bar"
endif
fwritecsvln(out,line)
enddo
return NIL
d) save, type bapp_w32c in the FAR Manager command line (build application w32 console,
it's one of the ccc builders)
e) type fuckthiscsv this.csv
My question is: do you have any quicker solution for it? Any programming
language, that is so quick to type, but not as unreadable as PERL?
Now, why would such things happen at all? This situation sounds like
a fucked-up project. Normally, you would know how the accounting software
accepts data. Normally, you would do this in Excel, simply autofilter and
and change by hand, or write a quick if(). Why don't I do that?
It's a problem of economics. If you don't live in the USA, but in Hungary,
then there is simply not enough money in the economy, and because time is money,
you simply don't have time to do things right. You don't have time to know
how the accounting software works, because you are pressed by your boss to
go and do sales instead of testing, because the company needs money. You don't have
time to write a correct CSV data manipulation software in 2 weeks in C,
because there is money for that. It even stands for Microsoft Hungary. They don't
have money to hire enough developers to localise Excel correctly, so if you
open a CSV file which contains "3.12", then Hungarian Excel will guess it's a
date, and will convert it to 12th of March, and if you format the cell
back to numeric, you will have something like 20328 or anything but the original
amount, so you don't want to use Excel. Maybe these problems can be avoided
by Regional Settings or anything, but you don't have time to test it out. The
biggest problem is you don't have time to try to foresee such problems, all of
these because time is money, money is not enough.
So you have to be creative, trust in luck, and write programs really, really quickly.
Which goes back to the original question: which is your language of choice for
such tasks? You can't use traditional languages like C/C++, Java, or such,
because these are designed for long, lazy, well-designed, well thought-out
application building. Perl is too complicated, you don't have time to learn
the syntax.
I prefer CCC.
And you?
Wild Wild East
Friday, May 28, 2004
Despite your protestations, judging by the length of your post, you don't seem like someone who is in any way pressed for time...
aeo
Friday, May 28, 2004
If it's just quick file readings or conversions, I use C++ (VC++ and the "hello world" console app wizard) and my various libraries I've made for myself over the years.
Cubist
Friday, May 28, 2004
"Perl is too complicated, you don't have time to learn
the syntax.".
Uh, what if you already know it? is that cheating?
vince
Friday, May 28, 2004
"I wonder why all your articles are about more or less big software projects developing some end-user applications from scratch."
I think that's mostly because problems with software development tend to increase exponentially with the size of the application. Small applications, little utilities, are just much easier to do. Not that they don't have their own set of problems and interesting design methods But developing them doesn't require anywhere near the foresight, planning, and discipline that are required when developing a larger application.
Herbert Sitz
Friday, May 28, 2004
Python is nice for that sort of thing.
Not really sure what the point of all that was though... you lack time.. so... you must do things fast. Okay. I gotcha.
Matt
Friday, May 28, 2004
http://www.gnu.org/software/gawk/gawk.html
Oren Miller
Friday, May 28, 2004
Ruby is like Perl, but better (to keep it short). Ruby supports OO, regular expression, DBI, OLE Automation, Win32API, etc. Do you want a .EXE? Use EXERB or something like it and you can have one. But if I needed performance I would have to use Delphi :-) Although you can find many of the characteristics of Ruby in Python or Perl. By using some custom scripts you can automate many things. Launch those scripts from your preferred editor and you're set.
Sadly today isn't a good day to advocate Ruby, because its main site is down.
Dewd
Friday, May 28, 2004
I used to do that kind of think in awk. Then I learned Perl and thought that was the ultimate. Then I learned Python, but sometimes I still use Perl.
Tom H
Friday, May 28, 2004
Oh, BTW, in case you were wondering, this is what it would look like in AWK.
if($8=="foo") $3="bar"; print
Have a nice day.
Oren Miller
Friday, May 28, 2004
Yes, answer:
Use python. So good.
Roose
Friday, May 28, 2004
Solution with Ruby:
require 'csv'
File.open('coolio_csv_fixed.txt','w'){|f|
CSV.parse('coolio_csv.txt'){|row|
if row[7].to_s == 'foo'
row[2] = 'bar'
end
f.puts row.join(',')
}
}
Dewd
Friday, May 28, 2004
You can do 90% of darn near anything with Korn Shell, awk and sed. The other 100% can be done with Perl. OK, so Perl can look like line noise (in the hands of an expert), but Python is really cool. It even enforces good formatting (space matters).
I also did some amazing stuff with REXX, back in the days of steam-powered mainframes...
The bottom line is that you don't need anything obscure - all the tools you could possibly need are already out there and they are free. Of course, you may have some Dilbert PHB managers who think that you have to use "mainstream" tools like Visual Braindead, or .NUTS, but you can always smile, nod in agreement, and use the right stuff anyway.
Puff The Magic Dragon
Saturday, May 29, 2004
Re: perl looks like line noise, how is that bad?
Matthew Lock
Saturday, May 29, 2004
The best thing about python in this kind of situation is the interactive interpreter. You can just start typing in python statements, load files in to variables, look at whatever variables you want; you're free to experiment, which is usually the quickest way to hack something together.
This will work with any language that has an interactive shell, but combined with Python's simple syntax and very powerful standard library you've got a killer combination.
Chris Tavares
Saturday, May 29, 2004
For this type of chore I like to whip something up in MASM or my Palm OS emulator.
Ryan Seacrest
Saturday, May 29, 2004
Personally, I think you can't beat an original Turing machine (symbols on paper). I usually whip one up in VHDL, download it to an FPGA, and bingo... It might take slightly longer to get the FPGA doing what you wnat, but then it's blindingly fast.
DSP's can be good, too, if you're in a hurry
hardware guy
Saturday, May 29, 2004
CCC looks a bit like all the rest of them, really. For "projects" of this size, I doubt any of them have any significant advantage over another, if you average it out over a number of throwaway scripts.
Python is slowly becoming more of a 'serious' language, but it does a good job in the throwaway department. I think the CSV example would look like this, but this is a bit of guesswork because it seems the CSV module has changed beyond recognition since last time I used it.
import csv
input=csv.reader(open("test.csv","rb"))
output=csv.writer(open("test2.csv","wb"))
for line in input:
out=line[:]
if out[8]=="foo":
out[3]="bar"
output.writerow(out)
You could do the above from the interactive shell quite easily, in fact the interactive mode is definitely one of python's best features.
Tom
Saturday, May 29, 2004
I would count myself lucky if I had a turning machine. I actually would build a one off electronic machine to do this kind of work. Coders these days don't know they're born.
Matthew Lock
Saturday, May 29, 2004
Whoa, cowboy. ;) Nice to see some passion and code; I don't know why people criticize your long post, long posts are faster to write than short ones.
Python has access to Java through Jython, and its normal libraries seem to get the job done even on Windows, for things like emails and stuff. I've delivered software with this tool, so I'm free to recommend it. But it is no longer my personal choice.
Ruby turned me off initially because of a Perlism which didn't seem to fit, but that didn't keep me from reading about it, and it seems the language's rules are more powerful than Python's.
Common Lisp is my personal choice, because it respects things like that Awk example Oren gave; it's a multiparadigm language which can mimic the best ideas of other languages, without forcing an aesthetics.
http://groups.google.com/groups?selm=038D36424A1C01F9.65E29BB8B196CA2C.A6955F3341EEC795%40lp.airnews.net&rnum=10
However, I don't "recommend" it, because anyone who learns it will be in the early-adopter category, and therefore a mention, not a recommendation, is most appropriate.
Tayssir John Gabbour
Saturday, May 29, 2004
I can write the same program in C++ or Java in about the same number of lines, typing it in as fast as I can type -- it's maybe a 20 or 30 second function to write in either language. If it takes you much longer than that, I'm inclined to think the problem is your code-foo, not the language.
(When I was writing Lisp every day, it probably would have been just as fast for me in Lisp. This task is simple enough that unless you're using a really low-level language like C or assembler, it's not going to take more than a 30-40 seconds to write.)
I don't know perl very well, but this "CCC" code doesn't look any simpler than perl code to do the same thing. As has been pointed out, the corresponding awk code is much simpler.
Are you looking for justification? Pity? Do you want us to say "There, there, it's ok. Go ahead and write crappy code in a no-name language, it's ok, we know you're right"?
i like fast women
Saturday, May 29, 2004
If the Clipper tool thingie works for you, then use it.
Personally I always use Ruby. Pyhon never quite worked for me. I don't like the way it does objects. Ruby fits like a glove.
When your dealing with these quick and dirty fixes then you should use the tool that works best for you. No amount of fancy language design will every compensate for your own experience.
Ged Byrne
Saturday, May 29, 2004
<?php
//semi-orthodox way:
$fp = fopen('file.csv', 'r');
while ($p = fgetcsv($handle, filesize('file.csv'), ',')) {
($p[7]=='foo')?$p[2]='bar':'';
$data .= implode(',', $p)."\r\n";
}
file_put_contents('file.csv',$data);
//my ridiculous one-liner
file_put_contents('file.csv',
implode("\r\n",
array_map(
create_function('$line',
'$p = explode(",", $line);
($p[7]=="foo") ? $p[2]="bar":"";
return implode(",",$p);'),
explode("\r\n",
file_get_contents('file.csv')
)
)
)
);
?>
josheli
Sunday, May 30, 2004
I guess the real answer here depends on what PC you are on.
If I was on my personal computer, then I would likely fire up ms-access, and link to the table (assuming it has headings in the csv file). So, after linking, I then whack ctrl-g in ms-access and then type in:
Currentdb.Execute “update tbltext1 set f3 = ‘ba’” where f8 = ‘foo’”
I count zero lines of code with the above.
Of course…lets assume we are on a client pc. (after all, you whole point here was that you don’t want to have rely on some special huge development IDE thing. I mean, if I am on my pc..that I have a zillion tools..and I will use the tool closest to my fingers.
Remember…we are also trying to save the most amount time. So, if I am on my client pc, I would look to see what application is running, Word, Excel, PowerPoint, Visio
Anything with VBA built in would do the trick. Lets assume word is loaded and running.
Whack alt-f11..Insert module. Erase the option explicit, and then type in the following:
Sub convit()
fin = FreeFile()
Open "c:\input.txt" For Input As fin
fout = FreeFile()
Open "c:\output.txt" For Output As fout
Do While EOF(fin) = False
Line Input #fin, buf
v = Split(buf, ",")
If v(7) = "foo" Then
v(2) = "bar"
buf = Join(v, ",")
End If
Print #fout, buf
Loop
Close
End Sub
Just place your cursor anywhere in the above code..and then whack f5. You are done!
The above code will run in ms-access, word, excel, and even VB6 if you go to all the trouble to load up such a large system. Anyway, my point here is what happens if you don’t have your cool clipper library? For example, for my ftp work, I always use IE6 BECAUSE is it ALWAYS there on each pc I visit! If you flip IE6 into folder mode, then you can’t even tell the difference between a local window, and your ftp window. It is rather great. (if you are behind a NAT router..then change the ftp mode into passive),
Since ALL computers have this nice ftp client, then why not learn that one? (there is a zillion ftp programs on the net..but why not learn the one that is ALWAYS on each pc?).
The same goes for the above VB example. However, let assume that you don’t have word, or excel loaded (remember, ..I said loaded....and, maybe these office program are not installed..but my point is I am TOO LAZY to even load up one of the office programs is THEY ARE NOT yet loaded. Remember, we don’t want to waste that 10 seconds to load up word, or Excel, or ms-access.
To do your above conversion, I would actually just right click on the desktop. Create a text file call myconv.txt. I would type in the following window script:
set fsin = CreateObject ("Scripting.FileSystemObject")
set txtIn = fsin.OpenTextFile("c:\input.txt")
Set fsout = CreateObject ("Scripting.FileSystemObject")
Set txtOut = fsout.CreateTextFile ("output2.txt")
do while txtIn.AtEndOfStream = false
buf = txtIn.ReadLine
v = split(buf,",")
if v(7) = "foo" then
v(2) = "bar"
buf = join(v,",")
end if
txtOut.WriteLine buf
loop
I would then save the text document, and then while the text document is highlight, whack f2, and re-name the file extension to .vbs (so we now have a file called myconv.vbs). You will then see the text document icon change to a windows script.
Just click on it…and your file will be converted.
So, why not use windows scripting, as NO IDE needs to be loaded. No software with the except of notepad. And, the code is VB code…likely the most popular language in the world.
So, I have choice of virtually any office program that HAPPENS to be running to solve my problem.
And, if nothing is loaded then all I had to load up was note pad.
So, to me, the above windows scripting solution is about the best since no software system needed to be loaded. Why not use what every windows desktop has to make this conversion? Why download pearl, ruby, php…or some strange language like lisp? (I am at such as loss here!). I mean, yea..if you are on your way cool development pc..then use any tool you want. However, when you get away from your development pc…then my solution starts to make the others here look downright silly…
Why not use plain old plain Jane windows to do this conversion? What happens if you don’t have your clipper system, or c++ compiler loaded?
Do not people learn how to use windows these days anymore?
The above solution seems to be about the best. I can walk into any clients computer and accomplish the above ON THEIR systems. Likely every windows person reading this post can use the above. I can’t say that for some clipper stuff, nor ruby, or insert your favorite IDE and language here!.....
Just use the stuff built into windows to do this!!! I am always amazed that some people come up with such amazing solutions when the basic windows scripting can do this stuff with ease?
Do you actually think a good Unix admin would fire up a ide to do the above task?
Albert D. Kallal
Edmonton, Alberta Canada
kallal@msn.com
http://www.attcanada.net/~kallal.msn
Albert D. Kallal
Sunday, May 30, 2004
Not nice, Albert. ;)
(Quick point -- myconv.vbs seems to have a problem. You're splitting a line on commas, but a CSV file may have "escaped commas" which shouldn't be splitted. Further, I hear different CSV files actually have different escape conventions, so Access may not handle them all. This isn't an attack, just pointing it out.)
I fully believe that a real programmer should be comfortable with different languages. People laugh at the language you champion, VB, yet you represent lisp as some pointless language. But I think both have their advantages, just assymmetrical ones. Power is having your cake and eating it.
With lisp, I have achieved far better reuse than other languages. You implemented a simple read/transform/write pattern. With lisp I can exploit patterns, as well as mimic better languages for the task like Awk.
However, I don't recommend lisp, merely mention it, because it's still in the early-adopter phase.
Tayssir John Gabbour
Sunday, May 30, 2004
I agree I was tough! Here!
My only point here was the question was NOT made in terms of using a large, or well defined IDE. The whole pretext of the question was quick and dirty…and may I say cheap also!
The question was phrased in terms of low budgets, and low time, and low cost development tools. It is in this light I fired some bullets here! (my apologies).
It just seems to me that you can use windows to do this..and not purchase, or load any tools. (I kind over make this point ;-)
And, no, I don’t take aim at lisp, or any of scripting languages (I think all should take a crack at learning lisp). I might debate if you should learn a stack language like forth!
However, I in NO way meant to take aim at PHP, or lisp. This are well established tools in our industry..and I would be really crazy (arrogant) to shoot them down! All sys adims use such tools of the trade..and should proudly do so!
As for escaped chars..hum..well I guess that depends if the data will have “,” inside the fields. But then again..I suspect many of the solutions posted will have trouble with these.
Sorry for shooting a bit too hard…
Sincerely,
Albert D. Kallal
Edmonton, Alberta Canada
kallal@msn.com
http://www.attcanada.net/~kallal.msn
Albert D. Kallal
Sunday, May 30, 2004
Unix shell tools.
UnxTools for Windows - same grep, sed etc.
and Perl.
Nekto2
Tuesday, June 1, 2004
 Recent Topics
Fog Creek Home
|
