Lazy DOD

my $fh = open 'foobar';
main_program_which_has_nothing_to_do_with_fh();
print $fh: "Done\n";

Under the current Lazy DOD, every single scope exit in the main
program will suffer.

So, how about we have a similar flag on PMCs to replace the current
one (even if it's named the same thing :-) which will trigger a DOD
run whenever *a* reference is lost to this PMC. This allows code that
doesn't need timely destruction (even if there's still a variable
around that wants it, which will probably *always* be the case) not to
pay for it.

The costs of such a thing require checking a PMC flag in every op
which has an out pmc argument. Is this acceptable?

Luke

I don't think so. Wyh should the perl6 compiler insert lazysweep opcodes
in code, where no filehandle is used? When no filehandle is used, no one
can get out of scope, so...

It could be different, when $fh goes into main_prog... but, as $fh is
used after main_prog again, its also senseless to insert lazysweep
opcodes.

IMHO too expensive.

> Luke

leo

Really ?

sub foo (%hash) {
...
delete %hash{value_is_a_filehandle};
...
}

Not a filehandle in sight.

Just because code does not explicitly use a filehandle does not mean that one
will not be affected by its actions. If you want timely destruction of filehandles
the you must call DOD on every scope exit, unless you know that a scope cannot
affect anything outside the variables it introduces itself, which are very rare

I have yet to see an inexpensive solution to timely destruction.

Graham.

Plus, since Perl is untyped, it has no idea whether something is or
isn't a filehandle at code generation time.

So, my DOD on loss of reference might get this case, if the filehandle
was loaded into a register before being deleted.... Then when the
register lost it, it would DOD.

There are, of couse, cases where this fails (as, I believe, any
timely destruction method without refcounting (and even refcounting
can't handle circular thingies)). But the idea behind this, er, idea
is that it will get 95% of the cases correct, and let a regular DOD
catch the other 5% (eventually).

Unless you're a good functional programmer... but that's not the
average Perl hacker.

I'd personally trade timely destruction in favor of speed, but I also
see an argument for strong semantics. Whatever's best in they eyes of
the king...

Also, as I've said before, the problem with filehandles (neglecting
IPC) is trivially solved by adding a DOD before each C<open>. But
it's no good as a general solution.

Luke

Still my second argument counts, I think. As $fh is used in line 1 and
line 3 the filehandle can't get unused in the main routine in the
middle. Untyped or not, $fh is known to hold a filehandle, because its
used as the LHS of an "open".

But of course for a slightly more complicated example the compiler will
be unable to follow the usage of the filehandle so that the lazysweeps
(or refcounting or whatever) are necessary.

> Luke

leo

Following the type is a halting problem ... which is why some of the
languages like JVM and IL enforce that all paths to a location produce
the same local & stack types ..

IMHO the best option here would be to let the programmer close the file
handle explicitly and else to catch them in lazy sweeps .

"If you want timely destruction , do it yourself"... ;-)

Of course , it might be just that I hate things which are hard to
control or trace and debug...

(lacking in parrot jargon , I assume a DOD is the counterpart of a
Finalize run, correct me if I'm wrong)

Gopal
--
The difference between insanity and genius is measured by success

If you want a fast program ... probably. As Perl defines, that files are
closed on scope exit (when the last reference to the file handle gets
out of scope), perl and parrot must have some means to handle this.

DOD = Dead Object Detection. When a PMC is found unused ("dead") and has
the active_destroy_FLAG set, then the destroy handler gets called, which
frees PMC specific memory or closes a file or shuts down a timer. This
is similar to the destructor in C++. A DOD run normally is not done on
scope exit, but when there is a shortage in some resources.

> Gopal

leo

Let me get this straight ... "on" scope exit ? or just "after" scope
exit ? ... And if it's possible , how does Perl 5 handle this sort of
behaviour ?.

I had a look to see if it really works that way with perl5 ...

open $somefile,"<test" or die;

And did an strace of perl5's run

read(3, "open $somefile,\"<test\" or die;\n", 4096) = 31
read(3, "", 4096) = 0
close(3) = 0
munmap(0x40027000, 4096) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
open("test", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fcntl64(0x3, 0x2, 0x1, 0) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
_exit(0) = ?

I could not find where the "test" is closed ..

Yes, a finalizer ... Like , I could have a finalizer and a deallocator
of which the finalizer is called before deallocation by the garbage
collector . Running around and calling the destroy handler is exactly
what I meant.

Another TLA entered into database :-)

> leo

Maybe, yep.

Nope, unfortunately not. I think everyone's gone over the problems,
plus it puts the burden on sections of code that don't guarantee any
sort of timely destruction, which I'd like to avoid.
--
Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk

Love to, but we can't. Perl 5 effectively guarantees timely
destruction. If Larry mandates that for Perl 6, then we have to do
it, like it or not. Or at least provide a way that perl 5 and 6 code
can make it happen, which I think the scheme I checked in does. The
fact that it can be turned *off* is the nice bit. The fact that full
sweeps can be darned pricey is the not-nice bit. A generational GC
scheme might make this better, I'm not sure.

It's when we sweep the arenas looking for objects that are unreferenced.

It's part of scope exit. When the scratchpad for the lexicals gets
deleted on scope exit (because exiting decrements its refcount to
zero) all the refcounts of the variables in the scratchpad get
decremented, and any of those that go to zero get cleaned up.

Perl generally doesn't do anything special when you fall off the end
of the world. It's a shortcut, of sorts. Try throwing the code in a
block, making sure you use a lexical filehandle, and see what happens.

#v+

$ strace perl -e '{ my $f; open $f, "1" } print "ok\n"'
...
open("1", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=3773, ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
close(3) = 0
fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 3), ...}) = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40017000
ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0
write(1, "ok\n", 3ok

#v-

I don't know, what perl5 does, when dying, but this isn't the normal
case.

leo

True. But that main routine could be in a different module that was compiled
seaprately. So the compiler at that time has no idea if what it is doing
will affetc filehandles, so would have to insert DOD runs just in case.

Graham.

To get back to nearly where perl 5 is, we could also hide the POST block
for sockets down in the Socket module, and most people would just see

use Sockets autoclose => 1;

Yeah, this doesn't "solve the problem" -- I don't think there's any
solution -- but it hopefully shows that it's not nearly as pervasive as
people seem to think.

/s

Yep. One of the compiler switches will disable strict end-of-scope
checking, so modules that don't care don't have to emit the code to
do the checking.

I think what we're going to have to do is have a way to mark
filehandes as either eager for destruction or lazy for destruction.
(I'm not sure which, it depends on the default Larry chooses) That
way there'll at least be some way to mark a filehandle as not needing
immediate destruction, so that if there aren't any
immediate-destruction filehandles (or other objects) around then we
don't trigger the DOD with the lazysweep op.

[ Possible solution to the problem of timely destruction of file handles. ]

Don't forget (not to imply that you have) that whilst filehandles may
deserve special treatment, they are simply a specific example of a general
problem which also needs to be solved.

I hope I am not driving this discussion round in circles, but are there
any problems with simply adding reference counting only to those objects
which have destructors and/or which request it?

--
Paul Johnson - pa...@pjcj.net
http://www.pjcj.net

Yes. Search back through the archives, the discussion is in there.
(Might also be in the docs or FAQ--if not I think I may add it)

I think that would be a good idea.

Tim [who'll need to ponder DBI handle issues one day...]

What makes you so sure that the filehandle will live on until line 3?

sub main_program_which_has_nothing_to_do_with_fh() {
...
#Badly named, as it turns out.
$fh=open "> otherfile";
...
}

--Brent Dax <bren...@cpan.org>
@roles=map {"Parrot $_"} qw(embedding regexen Configure)

>How do you "test" this 'God' to "prove" it is who it says it is?
"If you're God, you know exactly what it would take to convince me. Do
that."
--Marc Fleury on alt.atheism

Even standard filehandles can be in the second case--consider a locked
file, for example. You can't just divide the world into "files" and
"everything else".

You are cheating ;-)
main_program_which_has_nothing_to_do_with_fh
^^^^^^^^^^^^^^^^^^^^^

But anyway, this was a special case, where the compiler could follow the
life range of the file handle. Normally we will need lazysweep opcodes.

leo

I would have hoped that simply putting "lazy" on any filehandle opened read
only on a regular file would be good enough as a win. And if anything
places a lock on that file handle then it turns "eager". Everything else
starts out eager, and (obviously) lazy handles are garbage collected
like memory.

But until we have real sized programs running I don't think we can tell,
so it's probably too early to get too far into thinking about this.
Time would be better spent helping Jürgen Bömmels with the IO rewrite.

Nicholas Clark