darcs

Issue 1899 Improve parser tests and quickchecks

Title Improve parser tests and quickchecks
Priority feature Status needs-implementation
Milestone Resolved in
Superseder Nosy List dagit, dmitry.kurochkin, ganesh
Assigned To
Topics Devel

Created on 2010-07-26.01:26:43 by dagit, last changed 2010-07-26.17:23:15 by ganesh.

Messages
msg11863 (view) Author: dagit Date: 2010-07-26.01:26:42
See patch318 for context.

Having some round trip quickchecks would be great.  From this effort to 
date (I had a few failed attempts at this refactor), I think these are some 
of the properties that ideal tests would check:

  1. Basic sanity checks, can things be parsed in the ideal/typical cases?
  2. How do high level parsers compose?  Since the parser monad is stateful 
on the input, it would be nice to check that the parsers do not consume 
more/less at the end of their parse than they were meant to.
  3. Sequences are dealt with correctly: empty sequences, many elements, 
good elements followed by partial elements, etc.
  4. linesStartingWith and linesStartingWithEndingWith are hard to get 
right and performance critical.
  5. Dealing with almost correct input.  Like fuzz testing.

In my experience, #2 is important but hard to define.  The purpose of #5 is 
to see if we accept things that should be rejected.  I'm not aware of 
anyone using fuzz testing with Haskell, but darcs would be a good candidate 
for that type of testing.  The more tests we have to cover #4, the more 
aggressively we can refactor those for performance reasons.

http://en.wikipedia.org/wiki/Fuzz_testing
msg11865 (view) Author: dagit Date: 2010-07-26.01:29:42
I forgot to mention.  I'm assuming that many of the tests we could write 
would be round trip tests:  Randomly generate patchinfos/patches, show 
them, parse them back, and compare that they are still equal.

Checking the left over input after the parse is also of interest, as per 
#2.
msg11880 (view) Author: ganesh Date: 2010-07-26.17:23:15
Re #2, normally when you run the parser itself, it asserts that there is 
nothing left at the end of the input. So I don't think it's hard to test 
that precisely the right amount of input is consumed.
History
Date User Action Args
2010-07-26 01:26:43dagitcreate
2010-07-26 01:29:43dagitsetmessages: + msg11865
2010-07-26 10:24:51koweysetpriority: feature
topic: + Devel
2010-07-26 17:23:15ganeshsetnosy: + ganesh
messages: + msg11880