Fog Creek Software
Discussion Board




Need a text editor that can handle gig+ files

Anyone have any suggestions?  Preferably something with a free trial period.  My biggest file is 16.6 gig.

NeedAnEditor
Tuesday, August 24, 2004

WinVi

Jason
Tuesday, August 24, 2004

Good Gravy!  What the heck is in there?  I hope I never get an email attachment from you.

I usually use EditPlus, which has a free trial and does alright with megabyte files, but I don't have any gigabyte files to test with.

Whatever editor you use can't try to read the file and store it in a buffer.

Frank "Grimey" Grimes
Tuesday, August 24, 2004

Try http://www.emurasoft.com/

Pedro Vieira
Tuesday, August 24, 2004

Log files?

I've never had to edit anything quite *that* big, as I tend to preprocess with grep or Perl or something to reduce the amount of information I actually need to load into the editor...

John C.
Tuesday, August 24, 2004

Guys, I need to look at the file to figure out what I need.  I'll then use Perl to pull it out.  Thanks for the suggestions so far, keep 'em coming - I'll get around to trying them in the morning.

NeedAnEditor
Tuesday, August 24, 2004

more
less
head
tail

all classics... combine all of the above (well, more or less) to look at random lines of the file. then grep/findstr/perl.

mb
Tuesday, August 24, 2004

I have been able to successfully work with files in the range of several hundred megabytes with UltraEdit.

www.alexlechuck.com
Tuesday, August 24, 2004

SlickEdit does this.  Also very good for binary files.  I edit > 1 Gig binary files occaissionally (MPEG transport stream captures as well as other multimedia formats), and Slick works very well for this purpose.

hoser
Tuesday, August 24, 2004

Its a regular expression in perl, btw.

NeedAnEditor
Tuesday, August 24, 2004

Thanks for the SlickEdit ref, hoser.

IMHO, you need a viewer. The *nix tools will serve, if you know them.

If you're on winXX, Ztree ( www.ztree.com ) is a boon for those who can remember Xtree Gold. It will let you view and G(ather) snippets to put in your scripts. Does text searches too. Various presentation formats and filters - but knowing how to drive it is the trick.  30 day trial.

[WordStar never tripped me up with a filesize limit (if processing forwards) in the file, and I chopped up some biggish (4meg) billing logs with it. This is not a serious suggestion, more a historical comment.]

trollop
Tuesday, August 24, 2004

I'd suggest the free and clean SciTE editor ( http://www.scintilla.org/SciTE.html .)

GinG
Tuesday, August 24, 2004

Nobody mentioned textpad??  It's the bees knees.

It works with multi gig files and cost about US $30.

Free version at

http://www.textpad.com

lumberjack
Wednesday, August 25, 2004

On Unix/Linux, I think Fabrice Bellard's QEmacs fits the bill. A win32 port was started, but I'm not sure about its status.

Ori Berger
Wednesday, August 25, 2004

NeedAnEditor: "Its a regular expression in perl, btw."

Phew, I always knew those regular expressions were complicated, but growing them to gig+ files seems excessive ;-)

Nemesis
Wednesday, August 25, 2004

TextPad?  That's actually my favorite editor.  However, it completely choked on this 16 gig file, requiring a reboot.

NeedAnEditor
Wednesday, August 25, 2004

Multi-Edit is by far the best text editor I have ever encountered.

It integrates seamlessly with several compilers, color-codes many different programming languages, has columnar selection (including search/replace within the selection), conversion from UNIX to DOS, and probably a lot of stuff that would be more important to you that I don't even know about.

Seriously, that editor kicks ass. No, I don't work for them. I've used that editor for a little over six years though, so I am probably biased.

EditPlus I wouldn't recommend for extremely large files. AFTER the file is opened it works fine, but that seems to take much longer than it really should.

I am Jack's personal preference
Wednesday, August 25, 2004

I've used two editors in the past that have capabilities for "large files" (for sufficient values of large):

VEdit
UltraEdit

While both are fine and worked for what we needed, I liked UltraEdit better. Just an opinion, nothing to really justify it.

Be advised, with either of them, doing a "seach" or "search and replace" in gig + files takes a loooooooong time (coffee break time)

Sgt. Sausage
Wednesday, August 25, 2004

Scite is probably not a good bet for very large files -- it seems to uses more than 1 byte per char, presumably for all its syntax highlighting stuff. Judging by memory usage its memory usage is about 3x file size.

You could try ListXP ( http://www.listxp.com/ ), which is pretty useful. You can look at pretty large files, only tried up to 600MB myself but the home page mentions viewing a 56GB file so presumably it can do that too.

I should add that it only views files. You can't edit them. But if you're just looking for something in the file, that's probably sufficient.

Tom_
Wednesday, August 25, 2004

Oh, and if you search in list, it has to have scanned the entire file first. This took a while for the 600MB file, but nothing unbearable. My PC is pretty slow too. A modern one will probably work well.

Tom_
Wednesday, August 25, 2004

You could try Large Text File Viewer 2.0 ( http://www.swiftgear.com/ltfviewer/features.html ). It isn't pretty or user-friendly but it allows you to view sections of really large files. The .exe is only 432KB. I've used it and it worked where TextPad choked.

Matt
Wednesday, August 25, 2004

>
Guys, I need to look at the file to figure out what I need.  I'll then use Perl to pull it out.  Thanks for the suggestions so far, keep 'em coming - I'll get around to trying them in the morning.


I am wondering here.. I had been processing lots of data myself and like to know what kind of application you have that can give clue after looking at LOTS of data?

Others have suggested different tools to cut down the amount of text that you want to look at it and I think those are more approrpriate for any type of work (currently).

A Great Software Engineer
Thursday, August 26, 2004

I would suggest JWrite http://www.mwasoftware.co.uk/editor/index.htm
We use it at work to open large files in the sort of range you are talking about.

Shane Carroll
Thursday, August 26, 2004

*  Recent Topics

*  Fog Creek Home