[perl #42430] [PATCH] make :vtable imply :method

This patch makes the :vtable sub pragma imply the :method pragma,
which seems like common sense to me, since using :vtable without
:method doesn't make much sense, and vtable overrides are called as
methods from C anyway. Using :vtable and :method explicitly still
works, so no existing code is broken. It obviously affects design far
more than code - the patch is only two lines :).

--
Alek Storm

Actually, setting :vtable without setting :method makes a great deal of
sense, if you want to override a low-level vtable operation without
adding a named method to the class.

Allison

--
Alek Storm

> Just use the :anon flag. It was designed for this. Vtable methods are

Both options seem slightly wrong to me.

Vtable methods are methods, yes, so they should receive the invocant
automagically. I don't believe they do this currently:

.sub 'set_integer_native' :vtable :method
.param int size

.local pmc size_pmc
size_pmc = getattribute self, 'size'
size_pmc = size
.end

(runs fine)

.sub 'set_integer_native' :vtable
.param int size

.local pmc size_pmc
size_pmc = getattribute self, 'size'
size_pmc = size
.end

error:imcc:The opcode 'getattribute_p_ic_sc' (getattribute<3>) was not found.
Check the type and number of the arguments

Allison's right about method visibility outside of the vtable, and
throwing :anon on there seems a little hacky.

However, requiring :method to make vtable methods actually work if you use
self inside fails Allison's visibility problem.

-- c

Is this a behavior problem or a syntax problem? IIUC, putting :anon on a
sub makes it inaccessible only through the namespace - the sub itself
retains a reference to the namespace. How would you want it to work?

--
Alek Storm

<http://groups.google.com/group/perl.perl6.internals/browse_thread/thread/5bbf1eab0133d467?tvc=2>

I'm comfortable with a modification saying that :vtable always gets the
'self' parameter like :method does. But without :method, :vtable should
not make an entry in the namespace of the class, or call 'add_method' on
the class. This results in simpler syntax for the cases where you only
want to override the vtable entry.

# method name is the same as vtable name
.sub get_string :method :vtable

# accessible as either $obj.stringify() or vtable
.sub stringify :method :vtable('get_string')

# accessible only as vtable
.sub get_string :vtable

# accessible only as vtable, and sub name ignored
.sub stringify :vtable('get_string')

Allison

From the link: <<END
This is the main thing Chip and I talked about in our last face-to-face
meeting. We came up with 3 basic parameters: whether a method is a
vtable method, whether it has a vtable name distinct from the method
name, and whether it has a method name at all (or is anonymous, i.e.
only a vtable method). The interface I scrawled out over coffee is:

.sub get_string :method :anon :vtable
.sub stringify :method :anon :vtable('get_string')

Which reuses the existing concept of:

# method has an entry in the namespace
.sub stringify :method

# method has no entry in the namespace
.sub stringify :method :anon
END

This seems perfectly fine to me. The only change I think should be
made is that :vtable automatically sets :method to true. Using :anon
to exclude it from the namespace is what you originally proposed, so
I'm curious why you've changed your mind.

Making :method mean one thing when used with :vtable, and something
completely different without, seems like a really bad idea to me, and
is confusing to the user. The user will also be confused because
adding :vtable removes it from the namespace, which they didn't
explicitly ask it to do. What is :anon good for if we use a
completely different system for :vtable?

Here are the semantics you've proposed:

.sub get_string # normal sub, attached to a namespace but not a method

.sub get_string :method # a full method, attached to a namespace

.sub get_string :vtable # a vtable method, but not attached to a
namespace (user wonders why, since methods are attached to their
namespace)

.sub get_string :vtable :method # adding :method attaches it to a
namespace (*not* what :method means without :vtable - user wonders why
they need it, since it's already a method)

.sub get_string :vtable :anon # unspecified

.sub get_string :vtable :method :anon # now we're in trouble

The same thing, with what I've proposed:

.sub get_string # normal sub, attached to a namespace but not a method

.sub get_string :method # a full method, attached to a namespace

.sub get_string :vtable # a method that also overrides the vtable

.sub get_string :vtable :method # same as before

.sub get_string :vtable :anon # same, but not attached to the
namespace (note that the user has actually specified this, so they
expect it to happen)

.sub get_string :vtable :method :anon # same as before

--
Alek Storm

I don't see why this is confusing. Normal vtable methods (the ones defined in
C, for example) are not visible:

.sub 'main' :main
.local pmc my_int
my_int = new .Integer

my_int.'set_integer_native'( 10 )
$I0 = my_int

print "Integer: "
print $I0
print "\n"
.end

-- c

Vtable methods defined in C are visible from C. Therefore, it makes
sense that vtable methods defined in PIR are visible from PIR, at
least by default. Making :vtable imply :anon might be unintuitive to
users. Besides that, there's still the problem of :method meaning two
different things with that implementation. Basically, in a non-vtable
sub, there is the :anon flag to detach it from the namespace, but in a
vtable sub, it's detached by default, and the :method flag attaches it
to the namespace. Reversal bad. Continuity good.

--
Alek Storm

Of course, otherwise nothing would be able to call them.

That makes no sense to me. Are you saying that vtable methods defined in a
specific language should be visible to that language by default?

If that's true, then users have to *know* the implementation details of vtable
methods. Is it in C code or is it in PIR code?

That's precisely what vtable methods protect against! That's why they're in
vtables. That's why they're *not* visible as methods to PIR code.

I can't honestly think of a case where you want :vtable AND :method on a
vtable method, with the caveat that right now you *must* do that
because :vtable doesn't cause Parrot to include the invocant in the proper
register on calls.

Fixing that makes these problems all go away.

-- c

It was kind of a "duh" statement, but I was making the point that vtable methods in PIR should be visible by default.

I'm not entirely sure what you mean - I didn't mention languages. I meant that vtable methods are simply normal methods in a class, which can be easily hidden with the :anon flag. This is exactly the way it works now - I didn't propose we change anything besides making :vtable imply :method.

I have no idea why you're saying this. Why does making vtable methods visible expose their implementation details? I haven't said anything remotely like this.

I don't really see where you're coming from here. Vtable methods are callable directly from C, so there's obviously no problem there. Vtable methods in PIR are just overriding their C implementations, so it makes sense that they're callable directly as well. Also, they are easily hidden with the :anon flag. None of this is any different than what is already implemented.

I was talking about Allison's recommendation. If you read that, what I said will make sense.

Sorry for the miscommunication :)

--
Alek Storm

Vtable methods written in C are not visible to PIR code.

See the code example I posted.

If vtable methods written in PIR are visible to PIR code, I can draw no other
conclusion.

If I want to use a PMC and call a vtable method directly from PIR, I can do
that if and only if that method is a method defined in PIR. I cannot do that
if the method is a method defined in C.

Thus for my code to work robustly, I have to know about the internals of the
method and hope that it does not change.

Again, the only part of :method that :vtable needs to imply is the part that
sets the invocant in the proper register and makes 'self' available within
the body of the method.

-- c

True, but there is an increasing number of PCCMETHODs being written
that are just wrappers around a vtable method. Again, having the
vtable override visible is *optional*. Just like writing a PCCMETHOD
wrapper is optional.

I see what you were saying now - the user has to know whether a vtable
method was written in PIR or C to call it directly. There are two
choices involved here. The first is the choice of the user, who, by
calling it directly, has chosen to code for that specific PMC. Nobody
is forcing the user to do this; they can just as easily use some form
of indirection, just like before. The second is that the author has
the choice of providing a visible vtable method to the user, or not.
This is just like writing a wrapper around the vtable method in C or
PIR, only more convenient. No semantics are changed.

So, that still means :method can mean two different things, depending
on whether :vtable is present or not. That's still bad.

--
Alek Storm

Because I realized that under that system, you could never use :vtable
by itself (there's no such thing as a vtable without a PMC), and because
the most common case (only overriding the vtable entry and not modifying
the class namespace) required 3 modifiers on the sub. This way the 2
most common cases each only need one modifier (:method or :vtable).

The whole point of the original change was to keep vtable overrides from
polluting the namespace of the class. vtables are Parrot's way of
interfacing with PMCs at a low-level, and not every language will want
all the vtable behavior exposed. When they do, we give them the option,
but we don't expect most classes written in PIR to expose the vtable
entries as methods.

Just no effect, since :vtable already implies no entry in the namespace.

Just like ":method :anon", but also stores it as a vtable entry.

This works too, but doesn't optimize for the most common case.

Allison

How so? Without :vtable, :method adds a method to the class. With
:vtable, :method adds a method to the class.

Oh, I should add, under the new OO spec, methods don't get entries in
the "namespace" either, they only get entries in the class itself (since
you may have multiple classes with different implementations pointing to
the same namespace). It's really that fact more than anything else that
has changed my thinking on the appropriate division of sub modifiers.

Allison

But herein lies the problem. Saying ":vtable :method :anon" is just
like ":method :anon" doesn't make sense, because with this solution,
:method means something *different* when used with :vtable. ":vtable"
adds a hidden vtable entry, ":vtable :method" adds a visible vtable
entry, ":method :anon" adds a hidden method, so what does ":vtable
:method :anon" do? When used with :vtable, :method and :anon have
exactly opposite meanings, so who wins? I don't know, and it doesn't
matter who does win, because it's not obvious to the user. That is
why we can't use two opposite systems.

That's a good point. However, requiring two flags instead of one for
the most common case is not worth the costs I've outlined above and in
my previous posts.

I ran a test through all code in the parrot repository, and was about
to publish my results, before I realized that vtable overriding with
:anon wasn't working. I'll definitely fix this.

--
Alek Storm

:method doesn't add a method to the class - it makes the sub visible.
Two completely different things.

--
Alek Storm

I'd like to see some example code; this does not match my experience.

.sub 'main' :main
.local pmc my_int_class
my_int_class = subclass 'Integer', 'MyInt'

.local int int_type
int_type = find_type 'MyInt'

.local pmc my_int
my_int = new int_type

my_int = 100
my_int.'set_integer_native'( 200 )

my_int = 1.01
my_int.'set_number_native'( 2.02 )

my_int = 'hello'
my_int.'set_string_native'( 'world' )

.end

.namespace [ 'MyInt' ]

.sub 'set_number_native' :method
.param num value
print "Setting new value: "
print value
print "\n"
.end

.sub 'set_string_native' :vtable
.param string value
print "Setting new value: "
print value
print "\n"
.end

.sub 'set_integer_native' :vtable :method
.param int value
print "Setting new value: "
print value

You've misunderstood how the features work. Vtables and methods are
stored separately from each other. Both :vtable and :method say "this
code entity isn't a subroutine, and isn't visible as a subroutine".

Perhaps it would be clearer to you if we were using:

.sub
.endsub

.method
.endmethod

.vtable
.endvtable

Allison

That's because I was talking about what Allison proposed, not what was
currently implemented.

--
> Alek Storm

I understand entirely how both features work. I should, as I implemented
large sections of the code dealing with vtable overriding. Perhaps you
should check the code, because they are *not* stored separately from each
other. In the current revision, :vtable and :method used together stores
the sub as a vtable entry and as a visible method. Adding :anon detaches it
from the namespace. For the new PDD15 implementation, once #42406 and
#42407 are committed, I plan to rework how they're stored internally, but
that isn't relevant to our discussion of the interface.

In my previous posts, I was talking about what you proposed earlier, which I
won't go over here, because it's in your earlier post. If I didn't make the
distinction clear, I apologize.

--
Alek Storm

Yes, that's how they're implemented now. Currently, in order to use the
common feature of overriding the vtable entry without polluting the
namespace you have to specify ":vtable :method :anon". Which is huge
progress over what we had before (you used to override a vtable by
creating a .sub prefixed with "__").

I'm talking about how they will be implemented for PDD15. Methods and
vtables will be stored separately, because it will be quite common to
have vtable entries with no corresponding method, and methods with no
corresponding vtable entry. The interface is intended to reflect the
same fact.

Allison

That's exactly what this patch fixes - it changes ":vtable :method :anon" to
":vtable :anon".

Now I see what this interface is intended to accomplish; I was unaware PDD15
was going to be implemented this way. I still don't like it, for the same
reasons as before, but there's nothing I can do about it, so I'll be happy
to help implement it. I want to get back to coding now.

In the future, please keep in mind that I don't comment on anything unless I
understand it. You've tried to explain things to me in several tickets that
I already fully understand. I am especially surprised you would try to
explain the current implementation of vtable overriding to me, since I
implemented a lot of it. Thanks.

--
Alek Storm

It's not at all intended to insult your intelligence. (We have a lot of
incredibly smart people working on Parrot.) It's purely a matter of
"communication is clearly breaking down somewhere, so let's take a step
back and figure out where our assumptions are different". The process
would be much easier if we were all telepathic, but then telepathy has
its own disadvantages.

Allison

And if vtables are anonymous by default, it'll make it possible to have
a vtable and a method, with the same name. I think it's more likely to
be a good thing than a bad thing, but I imagine it will at some point
cause a problem for someone, other than confusion.

No one ever said the logic of a virtual machine intending to support
all dynamic languages would be simple.