darcs

Issue 2556 apply fails with 'hGetBuf: invalid argument (Invalid argument)' for large patch bundle

Title apply fails with 'hGetBuf: invalid argument (Invalid argument)' for large patch bundle
Priority Status resolved
Milestone Resolved in
Superseder Nosy List mihaigiurgeanu
Assigned To
Topics

Created on 2017-10-10.11:01:53 by mihaigiurgeanu, last changed 2024-06-29.10:59:37 by bfrk.

Messages
msg19722 (view) Author: mihaigiurgeanu Date: 2017-10-10.11:01:52
darcs apply bundle.dpatch 

fails with the error 'hGetBuf: invalid argument (Invalid argument) for very 
large patch bundle (my patch bundle has 2.1G)

If I send the patch bundle on the standard input to 'darcs apply', everything 
works:

    type bundle.dpatch | darcs apply

I used darcs 12.2.5, 64 bit on Windows 10
msg19724 (view) Author: bfrk Date: 2017-10-10.13:42:28
Thanks for the bug report.

The sources for 2.12.5 to not contain any call to hGetBuf, so I guess
this comes from a library call.

Can you attach the output of

  darcs apply bundle.dpatch --debug

(might be large, so you should redirect it to a file). Not sure if this
helps but it may give us an idea where to look.
msg19747 (view) Author: mihaigiurgeanu Date: 2017-10-12.10:47:43
Unfortunately, there is not match in the debug output. It immediately fails with the above error:


    D:\Projects\OPTIMIX\OPTIMIX>darcs apply --debug decompile-produitsimulationdaoimpl.dpatch
    Beginning identifying repository .
    Done identifying repository .
    Identified darcs-2 repo: D:/Projects/OPTIMIX/OPTIMIX
    darcs: D:/Projects/OPTIMIX/OPTIMIX/decompile-produitsimulationdaoimpl.dpatch: hGetBuf: invalid argument (Invalid argument)
    
    D:\Projects\OPTIMIX\OPTIMIX>
msg19757 (view) Author: bfrk Date: 2017-10-16.09:28:59
The fact that there is not much in the debug output does in fact help.
It clearly shows that the error happens right when the patch bundle is
being read from disk. The function that does this is
Darcs.Util.External.gzFetchFilePS. We have to find out why it fails.
msg19799 (view) Author: bfrk Date: 2017-11-03.10:44:33
Sorry for the latency, I was pretty much distracted by other stuff.

gzFetchFilePS reads the whole file into a strict ByteString. It uses
mmapFilePS unless mingw32_HOST_OS is true as a preprocessor symbol.
Looking at the docs for System.IO.MMap I see that Windows is supported
as well. Perhaps we should remove the #if here and use mmap unconditionally?

There is one more detail where reading from stdin differs: in isGZFile
we seek and read off the number of bytes from the last 4 bytes in the
file and return an Int. I think this fails if the size is greater than
2^31. This is the code:

       else do hSeek h SeekFromEnd (-4)
               len <- hGetLittleEndInt h
msg24057 (view) Author: bfrk Date: 2024-06-29.10:59:37
Well, patch bundles are usually not compressed, so we fall back to 
mmapFilePS, which on Windows (currently, still) does a plain B.readFile. 
It is not clear to me why this should fail on a 64-bit system. Indeed, it 
does not seem to fail with any IO exception in the CI after I added a 
regression test for this issue, so this seems to have been resolved at 
some point or it was a bug (perhaps with libraries or ghc) on Windows 10. 
See

patch 9aa0a328b2f404891aa5e4dd0765d86ce3e18890
Author: Ben Franksen <ben.franksen@online.de>
Date:   Sat Jun 29 12:54:52 CEST 2024
  * add (succeeding) regression test for issue2556
History
Date User Action Args
2017-10-10 11:01:53mihaigiurgeanucreate
2017-10-10 13:42:30bfrksetmessages: + msg19724
2017-10-12 10:47:44mihaigiurgeanusetmessages: + msg19747
2017-10-16 09:29:00bfrksetmessages: + msg19757
2017-11-03 10:44:35bfrksetmessages: + msg19799
2024-06-29 10:59:37bfrksetstatus: unknown -> resolved
messages: + msg24057