Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Tcl - compiling expressions

1 view
Skip to first unread message

Will Coleda

unread,
Sep 29, 2005, 12:00:28 PM9/29/05
to perl6-i...@perl.org
Tcl's [expr] command now compiles expressions to PIR (before, it
would create an AST that it would then interpret when you wanted the
value.). Note: the language itself is still interpreted, this is only
one command in the language.

E.g: given a command like

while {$a < 10} {incr $a}

Originally, the while would parse the $a < 10 expression once, then
interpret it (walk through the AST and generate a value) each time.
Now, it compiles the expression to PIR once:

.pragma n_operators 1
.sub blah @ANON
.local pmc read
read=find_global "_Tcl", "__read"
.local pmc number
number=find_global "_Tcl", "__number"
$P0 = read("a")
$P0 = number($P0)
$P1 = new .TclInt
$P1=10
$I2 = islt $P0, $P1
$P2 = new .TclInt
$P2= $I2
.return ($P2)
.end

and then just invokes it when it wants the current value.


Amos Robinson

unread,
Sep 29, 2005, 2:21:50 PM9/29/05
to perl6-i...@perl.org
Very cool. Will have to look into it soon.
Are you still contemplating making the whole thing compiled?

Will Coleda

unread,
Sep 29, 2005, 5:41:19 PM9/29/05
to Amos Robinson, perl6-i...@perl.org
Yes, that's the plan, but the initial implementation isn't going to
be a compiler like most people would expect:

For example, something like:

while {$a < 10} { incr a }

while isn't language syntax. it's a command. So, this code would
result in creating two PMCs for the args (first arg is {$a <10},
second is {incr a}), and then lookup the while PIR .sub, and invoke
it with the two PMC args.

Future versions of the compiler will be able to cheat (presuming the
while builtin hasn't been overriden) and generate inline-able PIR,
which should improve performance.)

Regards.

Amos Robinson

unread,
Sep 30, 2005, 1:39:06 PM9/30/05
to perl6-i...@perl.org
Eeek. Yes, I forgot just how crazy a language it was. :-D

Will Coleda

unread,
Oct 2, 2005, 1:40:10 AM10/2/05
to perl6-i...@perl.org
The simple version of the compiler is now mostly done in my sandbox:

Failed Test Stat Wstat Total Fail Failed List of Failed
------------------------------------------------------------------------
-------
t/cmd_global.t 3 768 6 3 50.00% 2-4
t/cmd_proc.t 4 1024 11 4 36.36% 3-4 8-9
t/cmd_return.t 1 256 2 1 50.00% 1
t/cmd_string.t 3 768 57 3 5.26% 21 45-46
t/tcl_backslash.t 4 1024 35 4 11.43% 31-34

These break down into two classes of errors:

1) ``Pad index out of range'' {global, proc, return}

2) Data::Escape::String can't escape unicode {string, backslash}

I've opened a TODO ticket for #2: [perl #37321] Any takers on this
appreciated.

If I can get #1 working, I'll just TODO everything that depends on #2
and get this checked in.

I've attached the generated PIR output for the sample program, which
outputs "10"

set a 0


while {$a < 10} {
incr a
}

puts $a

It's functional but ugly. (Patches welcome once I checkin)

output.txt

Will Coleda

unread,
Oct 2, 2005, 1:52:04 AM10/2/05
to perl6-i...@perl.org

On Oct 2, 2005, at 1:40 AM, Will Coleda wrote:
>
> I've attached the generated PIR output for the sample program,
> which outputs "10"
>
> set a 0
> while {$a < 10} {
> incr a
> }
> puts $a

Ok, technically, it's not the output of a complete PIR program, it's
the concatenated output of several chunks. One of the goals, however,
will be to generate a standalone PIR program that can be compiled to
bytecode.

Regards.

Will Coleda

unread,
Oct 3, 2005, 4:57:05 PM10/3/05
to Perl 6 Internals
Committed: partcl is now now a compiler. Though some tests still fail
(leo fixed one of the unicode issues)

t/cmd_global.t 3 768 6 3 50.00% 2-4
t/cmd_proc.t 4 1024 11 4 36.36% 3-4 8-9
t/cmd_return.t 1 256 2 1 50.00% 1

t/cmd_string.t 2 512 57 2 3.51% 45-46


t/tcl_backslash.t 4 1024 35 4 11.43% 31-34

Jerry Gay was getting bored with all tests passing, so this is for
him. =-)

Thanks to a conversation on #parrot with leo, I have a lead on fixing
the problems with global, proc, and return: but I think the easiest
solution will involve completing one of the upcoming items below, so
committing as is for now. The unicode issues in the last two test
files above could use some time to track down. Ping me if you'd like
work on this.

Next few compiler related things to do:

1) provide inline-able versions of builtins. (on the short list:
anything in examples/bench.tcl)

2) Change the compiler to call those if they're available. (For
anything that has an inlinable version, we can replace the
interpreted version with a call to the compiled version.)... this
will temporarily brake some rename tests, but:

3) Keep a global counter that is incremented whenever a procedure is
created or rename'd. When generating the compiled code, key against
this counter - if it's changed, we must fall back to the interpreted
version. (which, although it starts out with a a call to the compiled
builtin, might have been replaced at that point.).

4) Provide a way to dump the total generated PIR code instead of
compiling & executing it.

Regards.

On Oct 2, 2005, at 1:40 AM, Will Coleda wrote:

> <output.txt>


>
> On Sep 30, 2005, at 1:39 PM, Amos Robinson wrote:
>
>
>> Eeek. Yes, I forgot just how crazy a language it was. :-D
>>
>>
>>
>>> Yes, that's the plan, but the initial implementation isn't going to
>>> be a compiler like most people would expect:
>>>
>>> For example, something like:
>>>

>>> while {$a < 10} { incr a }
>>>

Leopold Toetsch

unread,
Oct 3, 2005, 6:34:17 PM10/3/05
to Will Coleda, Perl 6 Internals

On Oct 3, 2005, at 22:57, Will Coleda wrote:

> 3) Keep a global counter that is incremented whenever a procedure is
> created or rename'd. When generating the compiled code, key against
> this counter - if it's changed, we must fall back to the interpreted
> version. (which, although it starts out with a a call to the compiled
> builtin, might have been replaced at that point.).

I don't see the point. Your compiler can emit, e.g.:

"while"(test, body)

You provide an appropriate implementation of a default "while"
function, in your default namespace.
Whenever user code overrides the "while" word ("set while $foo" - or
some such - I dunno Tcl syntax details), you store_global the new
implementation as a compiled function into the same namespace (or as a
lexical).

Done.

> Regards.

leo

Jerry Gay

unread,
Oct 3, 2005, 7:03:43 PM10/3/05
to Will Coleda, Perl 6 Internals
On 10/3/05, Will Coleda <wi...@coleda.com> wrote:
> Committed: partcl is now now a compiler. Though some tests still fail
> (leo fixed one of the unicode issues)
>
> t/cmd_global.t 3 768 6 3 50.00% 2-4
> t/cmd_proc.t 4 1024 11 4 36.36% 3-4 8-9
> t/cmd_return.t 1 256 2 1 50.00% 1
> t/cmd_string.t 2 512 57 2 3.51% 45-46
> t/tcl_backslash.t 4 1024 35 4 11.43% 31-34
>
> Jerry Gay was getting bored with all tests passing, so this is for
> him. =-)
>
i can't tell you how happy this makes me :)
even more tests failing on windows (r9318)...

Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------

t\cmd_global.t 3 768 6 3 50.00% 2-4
t\cmd_inline.t 1 256 3 1 33.33% 2
t\cmd_proc.t 4 1024 11 4 36.36% 3-4 8-9
t\cmd_return.t 1 256 2 1 50.00% 1
t\cmd_string.t 2 512 57 2 3.51% 45-46
t\tcl_backslash.t 4 1024 35 4 11.43% 31-34
Failed 6/47 test scripts, 87.23% okay. 15/450 subtests failed, 96.67% okay.


t/cmd_inline.t:2 is the mysterious "can't spawn" bug that only seems
to infest windows:

t\cmd_inline...........NOK 2# Can't spawn ".\parrot.exe --gc-debug
languages/tcl/tcl.pbc
C:\DOCUME~1\particle\LOCALS~1\Temp\cmd_inline_2.tcl": Bad file
descriptor at ../../lib/Parrot/Test.pm line 238.
# '
# expected: 'ok
# '
# '.\parrot.exe --gc-debug languages/tcl/tcl.pbc C:\DOCUME~1\particle\LOCALS~1
\Temp\cmd_inline_2.tcl' failed with exit code 255

~jerry

Will Coleda

unread,
Oct 3, 2005, 9:52:16 PM10/3/05
to Leopold Toetsch, Perl 6 Internals

That's actually how things work right now (very similar to the
interpreter from previous versions).

I'm suggesting switching to inlining the code when possible... and
then watching out for cases where the inlined version is stale (using
a fairly simple method, but one that'll still give us a boost.). The
degenerate case using this caching/inlining will be the current
implementation (plus an integer compare before each invocation)

Is the argument that inlining will not (even theoretically) provide a
speed boost? I imagine on single pass code it'd be a wash, but for
something like while, we're potentially avoiding one find_global per
statement per iteration of the body, as well as the associated call/
return for each element in the body. Even for something like examples/
bench.tcl, that'd be tens of thousands of parrot sub calls/returns
that we could potentially eliminate. as low as that overhead is, I
imagine it's got to add up.

The nice thing about doing the inlining _as it's available_ means
that (presuming this *is* a good thing), we can inline the builtins
that give us the best boost first, and I don't have to reimplement
everything all at once.

Regards.

Leopold Toetsch

unread,
Oct 4, 2005, 5:11:00 AM10/4/05
to Will Coleda, Perl 6 Internals

On Oct 4, 2005, at 3:52, Will Coleda wrote:

> On Oct 3, 2005, at 6:34 PM, Leopold Toetsch wrote:
>

>> I don't see the point. Your compiler can emit, e.g.:
>>
>> "while"(test, body)
>>

> That's actually how things work right now (very similar to the
> interpreter from previous versions).

Not quite:

$ find lib -name '*.pir' | xargs grep 'sub.*while'
lib/commands/while.pir:.sub "&while"

You obviously have to mangle the subname first, before you can call it.

> Is the argument that inlining will not (even theoretically) provide a
> speed boost?

No, not at all, that's fine. I just wanted to mention that there are
still a lot of code pieces left over from the interpreter.

> Regards.

leo

Will Coleda

unread,
Oct 4, 2005, 9:16:41 AM10/4/05
to Leopold Toetsch, Perl 6 Internals

On Oct 4, 2005, at 5:11 AM, Leopold Toetsch wrote:

>
> On Oct 4, 2005, at 3:52, Will Coleda wrote:
>
>
>> On Oct 3, 2005, at 6:34 PM, Leopold Toetsch wrote:
>>
>>
>
>
>>> I don't see the point. Your compiler can emit, e.g.:
>>>
>>> "while"(test, body)
>>>
>>>
>
>
>> That's actually how things work right now (very similar to the
>> interpreter from previous versions).
>>
>
> Not quite:
>
> $ find lib -name '*.pir' | xargs grep 'sub.*while'
> lib/commands/while.pir:.sub "&while"
>
> You obviously have to mangle the subname first, before you can call
> it.

Yes, this is mandatory, as subs and variables must co-exist in a
parrot namespace, but in tcl, you can have a sub and a var with the
same name in the same namespace. This _might_ change once Matt
Diephouse's proposal is formalized, but for now, it's the easiest
solution. (The other is mangle the namespaces instead of the names,
but I felt this solution would be better for language interoperability.)

I was focusing more on the dispatch part than the naming conventions
in my earlier reply, sorry.

>
>> Is the argument that inlining will not (even theoretically)
>> provide a speed boost?
>>
>
> No, not at all, that's fine. I just wanted to mention that there
> are still a lot of code pieces left over from the interpreter.
>

Oh, sure. I'm doing now the bare minimum to be called a compiler, I
think, but this particular piece is probably going to stay,
regardless of compile vs. interpret.

Regards.

0 new messages