Comparing Pythons

2004-10-04T16:07:32Z

Comparison of various Python implementations for conformance.

Methodology

I started with Mical's test cases as they were small micro unit tests.
I removed any test that failed using CPython 2.3.4. Some tests were designed to raise an exception, and unhandled exceptions are likely to vary between implementations.
I ran each test on pie-thon, ast2past, and pirate on the latest Parrot from CVS on Debian. I also ran each test on jython on Java 1.4.2 and iron-python on .Net 1.1.4322, both on WindowsXP.
I marked each output as pass if output as compared to the CPython implementation on the same platform was byte for byte identical.
I marked each output as pass* if the outputs differed only in leading or trailing spaces. There were a fair number of these.
All other outputs were marked as FAIL. Where possible, I tried to capture stderr.
Click on the links to see the source of the test and the outputs.

Results

test	cpython	piethon	past	pirate	jython	ironp
arg_order	pass	pass	FAIL	pass*	pass*	pass*
assignment	pass	pass	FAIL	pass	pass	pass
augmented_assignment	pass	FAIL	FAIL	pass	pass	FAIL
bitwise	pass	FAIL	pass	pass	pass	pass
break	pass	pass	FAIL	pass*	pass*	pass*
compare	pass	pass	FAIL	FAIL	FAIL	pass*
continue	pass	pass	FAIL	pass*	pass*	pass*
def	pass	pass	FAIL	pass	pass	pass
dict	pass	FAIL	FAIL	pass	pass	pass
euclid	pass	pass	FAIL	pass	pass	pass
except_called_raise	pass	pass	FAIL	pass	pass	pass
for	pass	pass	FAIL	pass*	pass	pass
for_else	pass	pass	FAIL	pass	pass	pass
generator	pass	pass	FAIL	pass	pass	FAIL
global	pass	FAIL	FAIL	pass	pass	pass
if	pass	pass	pass	pass	pass	pass
if_expr	pass	pass	pass	pass	pass	pass
instance	pass	FAIL	FAIL	pass	pass	pass
iterator	pass	FAIL	FAIL	pass	pass	pass
lambda	pass	pass	FAIL	pass	pass	pass
lambda_anonymous	pass	FAIL	FAIL	pass	pass	pass
lexical_scope	pass	FAIL	FAIL	pass	pass	FAIL
listcomp	pass	FAIL	FAIL	pass	pass	pass
logic	pass	FAIL	FAIL	pass*	pass*	pass*
math	pass	pass	pass	pass	pass	pass
methods	pass	pass	FAIL	pass	pass	pass
microthreads	pass	FAIL	FAIL	FAIL	pass*	FAIL
microthreads_more	pass	pass	FAIL	FAIL	pass*	FAIL
minimal_function	pass	pass	FAIL	pass	pass	pass
multi_return	pass	pass	FAIL	pass	pass	pass
nested_call	pass	pass	FAIL	pass	pass	pass
nested_subscripts	pass	pass	FAIL	pass	pass	pass
no_return	pass	pass	FAIL	pass	pass	pass
pass	pass	pass	pass	pass	pass	pass
print	pass	pass	pass	pass	pass	pass
range	pass	pass	FAIL	pass	pass	pass
recursion	pass	pass	FAIL	pass*	pass*	pass*
return_tuple	pass	FAIL	FAIL	FAIL	pass	pass
shift	pass	FAIL	pass*	pass*	pass*	pass*
simple_listcomp	pass	FAIL	FAIL	pass	pass	pass
simplemath	pass	pass	pass	pass	pass	pass
try_except	pass	pass	FAIL	pass	pass	pass
tuple	pass	pass	FAIL	FAIL	pass	pass
unary	pass	FAIL	FAIL	FAIL	FAIL	pass*
unpack_list	pass	pass	FAIL	pass*	pass*	pass*
while	pass	pass	FAIL	pass*	pass*	pass*
while_else	pass	pass	FAIL	pass	pass	pass