Performance of reading string to Int in Haskell ( Bytestring vs [Char])

Question

Just doing some simple benchmark on Bytestring and String. The code load a files of 10,000,000 lines, each an integer; and then convert each of the strings into the integer. Turns out the Prelude.read is much slower than ByteString.readInt.

I am wondering what is the reason for the inefficiency. Meanwhile, I am also not sure which part of the profiling report corresponds to the time cost of loading files (The data file is about 75 MB).

Here is the code for the test:

import System.Environment
import System.IO
import qualified Data.ByteString.Lazy.Char8 as LC

main :: IO ()
main = do
  xs <- getArgs
  let file = xs !! 0

  inputIo <- readFile file
  let iIo = map readInt  . linesStr $ inputIo
  let sIo = sum iIo

  inputIoBs <- LC.readFile file
  let iIoBs = map readIntBs  . linesBs $ inputIoBs
  let sIoBs = sum iIoBs

  print [sIo, sIoBs]

linesStr = lines

linesBs  = LC.lines


readInt :: String -> Int
readInt x = read x :: Int

readIntBs :: LC.ByteString -> Int
readIntBs bs = case LC.readInt bs of
                Nothing -> error "Not an integer"
                Just (x, _) -> x

The code is compiled and executed as:

> ghc -o strO2 -O2  --make Str.hs -prof -auto-all -caf-all -rtsopts
> ./strO2  a.dat +RTS -K500M -p  

Note "a.dat" is at aforementioned format and about 75MB. The profiling result is:

       strO2 +RTS -K500M -p -RTS a.dat

    total time  =      116.41 secs   (116411 ticks @ 1000 us, 1 processor)
    total alloc = 117,350,372,624 bytes  (excludes profiling overheads)

COST CENTRE MODULE  %time %alloc

readInt     Main     86.9   74.6
main.iIo    Main      8.7    9.5
main        Main      2.9   13.5
main.iIoBs  Main      0.6    1.9


                                                        individual     inherited
COST CENTRE   MODULE                  no.     entries  %time %alloc   %time %alloc

MAIN          MAIN                     54           0    0.0    0.0   100.0  100.0
 main         Main                    109           0    2.9   13.5   100.0  100.0
  main.iIoBs  Main                    116           1    0.6    1.9     1.3    2.4
   readIntBs  Main                    118    10000000    0.7    0.5     0.7    0.5
  main.sIoBs  Main                    115           1    0.0    0.0     0.0    0.0
  main.sIo    Main                    113           1    0.2    0.0     0.2    0.0
  main.iIo    Main                    111           1    8.7    9.5    95.6   84.1
   readInt    Main                    114    10000000   86.9   74.6    86.9   74.6
  main.file   Main                    110           1    0.0    0.0     0.0    0.0
 CAF:main1    Main                    106           0    0.0    0.0     0.0    0.0
  main        Main                    108           1    0.0    0.0     0.0    0.0
 CAF:linesBs  Main                    105           0    0.0    0.0     0.0    0.0
  linesBs     Main                    117           1    0.0    0.0     0.0    0.0
 CAF:linesStr Main                    104           0    0.0    0.0     0.0    0.0
  linesStr    Main                    112           1    0.0    0.0     0.0    0.0
 CAF          GHC.Conc.Signal         100           0    0.0    0.0     0.0    0.0
 CAF          GHC.IO.Encoding          93           0    0.0    0.0     0.0    0.0
 CAF          GHC.IO.Encoding.Iconv    91           0    0.0    0.0     0.0    0.0
 CAF          GHC.IO.FD                86           0    0.0    0.0     0.0    0.0
 CAF          GHC.IO.Handle.FD         84           0    0.0    0.0     0.0    0.0
 CAF          Text.Read.Lex            70           0    0.0    0.0     0.0    0.0

Edit:

The input file "a.dat" are 10,000,000 lines of numbers:

1
2
3
...
10000000

Following the discussion I replaced "a.dat" by 10,000,000 lines of 1s, which does not affect the above performance observation:

1
1
...
1

Answer 1

read is doing a much harder job than readInt. For example, compare:

> map read ["(100)", " 100", "- 100"] :: [Int]
[100,100,-100]
> map readInt ["(100)", " 100", "- 100"]
[Nothing,Nothing,Nothing]

read is essentially parsing Haskell. Combined with the fact that it's consuming linked lists, it's no surprise at all that it's really very slow indeed.