<p>I'd like to slice and dice large datafiles, up to a gig, in a fairly quick and efficient manner.  If I use something like UNIX's "CUT", it's extremely fast, even in a CYGWIN environment. </p>

<p>I've tried developing and benchmarking various Ruby scripts to process these files, and always end up with glacial results.  </p>

<p>What would you do in Ruby to make this not so dog slow?</p>

This question reminds me of Tim Bray's
Wide Finder project
. The fastest way he could read an Apache logfile using Ruby and figure out which articles have been fetched the most was with this script:
```
```
counts = {}
counts.default = 0

ARGF.each_line do |line|
   if line =~ %r{GET /ongoing/When/\d\d\dx/(\d\d\d\d/\d\d/\d\d/[^ .]+) }
     counts[$1] += 1
   end
end

keys_by_count = counts.keys.sort { |a, b| counts[b] <=> counts[a] }
keys_by_count[0 .. 9].each do |key|
    puts "#{counts[key]}: #{key}"
end
```
```
It took this code 7½ seconds of CPU, 13½ seconds elapsed, to process a million and change records, a quarter-gig or so, on last year’s 1.67Ghz PowerBook.