Perl grammar for Perl5 -> Perl6

I am Perl user from near year 0. For me the easiest way to learn (,
track, and get to the point of contributing to) Perl6 would be a Perl
grammar (a regex rule set in, preferably, Perl6) that transforms any
Perl5 script into a Perl6. Of couse, besides learning Perl6 for a
regex'r or Perl5'r such as myself, and tracking, and documenting 6, it
would have huge use for Perl5 users making or considering the
transition.

For example one can infer the structure and some of the rules from
http://svn.openfoundry.org/pugs/docs/other/porting_howto which is
however written out in informal (not regex) rules such as $array[idx] ->
@array[idx]

Is there such a Perl5->Perl6 translator underway?

Sincerely,
Peter Schwenn

www.schwenn.com

p.s. I'm not yet up to originating a good one, but believe I could make
contributions in extending and checking it.

p.s. A Perl5 starting point might be better since running it on itself
would provide and initiate at least 2 major test modes.

p.s. The developing form of such a grammar could likely lead to
a grammar package which facilitates rule sets for languages in
other domains, in terms of illuminating means of choosing among modes
for rule ordering, collecting, scoping, re-application, recursion,
exclusion and so forth.

IMHO machine translation is not that good a way to start learning -
the real benefit of Perl 6 is in the features which have no perl 5
equivalents and solve problems much more elegantly.

The best thing to do is to hang out on #perl6 and get involved with
the test suite, as well as reading the synopses.

Perhaps writing a toy program or something like that could also
help.

These are rules for humans with lots and lots of context info...
Furthermore, they are more of a perl 6 gotcha list, for perl 5
programmers than a translation guide.

Larry Wall is working on using the perl (5) interpreter to create
compiled output (as opposed to just something that executes in
memory) that can then be read by a translator without actually
parsing perl 5.

Before this happens this will be very very hard - the high level
language has vast amounts of implications on execution etc, but the
opcode tree is much more simpler to predict (for a computer).

Since perl 5's actual parser and tokenizer will be used for this it
won't be very extensible, but this is important because perl is
reaaaaaaaaaaaaaaaaaaaaaaaaallly hard to parse.

--
() Yuval Kogman <nothi...@woobling.org> 0xEBD27418 perl hacker &
/\ kung foo master: /me supports the ASCII Ribbon Campaign: neeyah!!!

http://dev.perl.org/perl6/
>> Perl6 FAQ <<

--
Grtz, Ruud

I started toying with one, but didn't get very far yet. PPI (on CPAN)
would be a good way to start, but it needs to be extended to be used
as a Perl5->Perl 6 translator.

Greetings,
Christian

--
cr...@web42.com - http://christian.web42.com - http://www.web42.com/crenz/

"If the Creator really did write himself into his own story, that's
what we ought to expect to see. Creative solutions. And this
creativity is intended to be transitive. We are expected to be
creative. And we're expected to help others be creative." -- Larry Wall

Except it would be lovely to have a smart enough refactoring translator
that it could recognize where those elegant solutions are possible and
at least give the option of attempting them. Or at least a hint that
there might be a better way.

: The best thing to do is to hang out on #perl6 and get involved with

Sure, but some of our toys are bigger than others. :-)

: > Is there such a Perl5->Perl6 translator underway?

Yes, I have a version of 5.9.2 that dumps out some *very* strange
XML that represents, as closely as possible, the exact meaning of
the code to Perl 5, along with all the syntactic bits. I then filter
that strange XML back into something approximating an AST. I am in
the process of proving to myself that I'm getting enough information
out of this to recreate the original Perl 5, so I jokingly call this
my Perl5-to-Perl5 translator. As of today, I'm able to translate
76.57% of the t/*/*.t files that come with the Perl distribution.
Considering that last week this number was down at about 5%, it would
seem that I've been making a lot of progress. But most of the work
went into that first 5%, and a lot of work will likely go into the
last 5% as well. To get that first 5% I basically had to completely
refactor the lexer and the grammar without changing anything, which
is of course impossible. The Perl 5 parser forgets or misplaces an
astounding variety of information that the translator needs, and
you can't just go and tell it to turn off the optimizations, because
in fact most of those optimizations are deeply interwingled with
semantic analysis and transformations as well.

Basically, every skipspace() in toke.c and every op_free() in op.c
and every rule reduction in perly.y loses necessary information.
To attempt to do what I'm currently doing you would have to be
completely insane like me. It's a total nightmare. If I were
Catholic I'd be hoping this all counts as pennance for my past sins,
and gets me out of 100 million years of Purgatory or so. But being
a Protestant, I'm merely repenting of my past sins, and thinking
about maybe repenting my future ones.

And if I were Jewish I'd've said "Oy vey" many times over. :-)

Anyway, once I get to 100% of the t/ files, I'll make it translate
all of CPAN back to itself. And at some point I'll take a first
whack at the Perl5-to-Perl6 translator, then open it up for community
participation. It's still just a bit too early for that, though,
because there's such a delicate interplay between refactoring bits
of perl without changing anything vs trying to guess whether we are
getting enough type and structural information out to recreate the
original in the backend. There are already more than 10000 lines of
code in the backend just to undo the damage done by the Perl 5 engine.

: Before this happens this will be very very hard - the high level

Right. But my intent is to write a really good translator, and that
implies that it has to be a multi-level translation. That involves
keeping track of all the subtle semantic and pragmatic information
as well as the basic syntactic information. Otherwise we might as
well just feed Perl 5 to babblefish and see if Perl 6 comes out...

: > p.s. The developing form of such a grammar could likely lead to

And it's oh about 20 times harder to tell if you've parsed it
correctly from a semantic point of view. Consider that every little
instrumentation tweak I've made to the lexer has had about a 50%
chance of inducing strange distortions in the meaning of "Perl".
I would be completely lost without the existing regression tests.
That is why I went for minimal instrumentation and try to undo most
of the damage in back end. (Or I guess it'll be a "middle end"
once we start translating to Perl 6.)

As for the original question, I think that the Perl 6 grammar will
be a much better example for how to parse other languages than a
Perl 5 grammar would be, since one of the underlying design currents
from the beginning has been that Perl 6 had to be a language that
was amenable to parsing by Perl 6 rules (with a little help from a
bottom-up operator-precedence expression parser.)

Larry

LW> And if I were Jewish I'd've said "Oy vey" many times over. :-)

or if you were a lazy jewish perl6 hacker you would code:

say "Oy Vey!" for 1 .. ;

:)

uri

--
Uri Guttman ------ u...@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org

Once Patrick is done with PGE, will it be able to parse Perl5? If so,
why aren't we focusing on that?

Rob

PGE isn't much of a parser; it's more like a framework for bulding
parsers. So, once PGE is done, to parse perl5, you'll still have to
write a grammar-ish thing that includes all of the maze of twisty
executions that perl currently goes through to "parse perl".

The same holds for parsing perl6 only the twisty maze isn't there (or is
at least much much smaller)

-Scott
--
Jonathan Scott Duff
du...@pobox.com

Sure, but if you want to parse it *correctly* and know what it actually
means, I estimate it'll take about 20 times as much effort as you're
thinking it will, because there are an incredible number of foibles
in the Perl 5 parser. Otherwise you might as well just use perl5:PPI.

I'm not trying to malign the PPI approach here. It works most of
the time, and when it works, it works fine. I don't doubt that it
will produce better translations of certain programs, especially
source-filtered programs where the filtering can essentially be
ignored. But with the "PPD" approach I'm aiming first for a correct
semantic translation and secondarily for a close syntactic translation,
so what you'll get out of my approach is a post-source-filtered
translation that is likelier to work but less likely to be readable.
For a known source filter we can perhaps then refactor that back to
a more readable form.

In the long run these different approaches will complement each other,
and help us better understand each approach's weaknesses and strengths.
It's just that I have to take the PPD approach because no one else
is simultaneously knowledgable enough and nutty enough to attempt
it and succeed. (Whether I am myself knowledgable enough and nutty
enough to succeed is yet to be decided. Recent signs are encouraging,
especially in the latter department.)

Larry

Hmm. Maybe...

Perl 5 is a twistly maze of passages all different.

Perl 6 is a twistly maze of passages all alike.

Larry

There are several source filters that make select changes and modules
that give select Perl 6 features to Perl 5. All in all, there's a
lot there -- I've written Perl6::Contexts, for example, which diddles
the bytecode to give Perl 5 programs most of the new contexts from
Perl 6. In fact, I kind of even wrote a book on doing this stuff...
http://www.perl6now.com. If you download the code listings, you'll get
examples of a couple dozen of these modules. As far as your example
of $array[idx], see Perl6::Variables. I have non-release version on
Perl6::Variables in my CPAN area, SWALTERS, that adds more of the
Perl 6 changes.

I know this isn't the approach you had in mind, but I thought I'd
mention it, as there is actually stuff there to play with. Fundamentally,
regexen to transform a Perl program aren't a bad idea, but this
approach, of fixing up Perl 5, is less likely to introduce bugs
into the code and keeps the production-ready environment that Perl 5 is.
Transforming the code isn't a bad idea where there are transformations,
but in many cases, there aren't. I suppose that could be a fatal error,
but people would still be trying to "convert" their Perl 5 programs,
much as when people where converting to C from B, and any failure
scenario would create a flood of help requests, bug reports, etc.

I hope this helps!

Best regards,
-scott

-scott

> _______________________________________________
> Phoenix-pm mailing list
> Phoen...@pm.org
> http://mail.pm.org/mailman/listinfo/phoenix-pm