Unable to open '/../../lib/Storable.pm' for reading: No such file or directory.

NYTProf version:
## $Id: NYTProf.pm 665 2009-01-05 23:05:37Z tim.bunce $
our $VERSION = '2.07';

Running on MacOSX, Perl 5.8.9

Most of my time is spent in Storable
(238 1 2 53.2s 53.2s Storable::::mretrieveStorable::mretrieve(xsub))

Detail
# spent 107ms within Storable::dclone which was called 125 times, avg
857µs/call:
# 125 times (107ms+0s) by Class::DBI::ColumnGrouper::clone at line 68
of /opt/local/lib/perl5/site_perl/5.8.9/Class/DBI/ColumnGrouper.pm,
avg 857µs/call
# spent 39.9ms within Storable::net_mstore which was called 42 times,
avg 951µs/call:
# 42 times (39.9ms+0s) by Storable::_freeze at line 339, avg 951µs/
call
# spent 53.2s within Storable::mretrieve which was called 237 times,
avg 224ms/call:
# 237 times (53.2s+0s) by Storable::thaw at line 415, avg 224ms/call
Unable to open '/../../lib/Storable.pm' for reading: No such file or
directory.

I've tried adding nytprofhtml --lib /opt/local/lib/perl5/5.8.9/
darwin-2level (where Storable lives), but this makes no difference?

Any suggestions on where to go from there?

Off the top if my head, and without access to the code, I'd guess the
path to Storable.pm is the problem. The prefix "../../lib" added to
the library include path you provided would result in something like
"perl/lib/lib/"

It's been a while since I looked at this stuff and I am on a train.
Maybe that will help you debug.
--
Adam J. Kaplan
Duct Tape Specialist & Software Engineer
The New York Times Company

It's caused by AutoSplit being run on Storable.pm before it's installed
by perl. AutoSplit adds #line directives to the auto-split files but
those #line directives have relative paths that bear little relation
to where the files end up.

It's probably possible to work-around it. If someone wants to volunteer
I'd be happy to guide them.

Tim.

I have just built a CentOS box, and am having the same problems.
I'd be happy to volunteer what little skills I have.

Anyone have a workaround for the Storable issue yet? (I have the same
issue with a fresh 5.10 install on OS X 10.4.11)

I am able to generate an nytprof.out file, but get a similar Storable
not found error when trying to run nytprofhtml.

Tim - any guidance welcome.

--
Will "Coke" Coleda

You and 'tcbarrett' seem keen, great!

Here's a summary of the problem and what we can do about it...

AutoLoader (or rather, AutoSplit, http://search.cpan.org/perldoc?AutoSplit)
writes an autosplit.ix containing a bunch of stubs, and a bunch of
foo.al files, one per AutoLoad'able sub. AutoLoader loads the autosplit.ix
file for the module that's 'use'd it, and supplies an AUTOLOADer that
loads the foo.al for whatever sub is used.

From NYTProf's perspective, with trace=2, we see this sequence of fids
(file id's) being assigned:

> New fid 2 (after 1:7 ) 2 e0:0 /usr/lib/perl5/5.8.8/i386-linux-thread-multi/POSIX.pm
> New fid 5 (after 3:159 ) 2 e0:0 /usr/lib/perl5/5.8.8/i386-linux-thread-multi/auto/POSIX/autosplit.ix
> New fid 6 (after 3:961 ) 2 e0:0 ../../lib/POSIX.pm /tmp/t/../../lib/POSIX.pm

(there are two paths on that last line because the filename perl has is
relative, so NYTProf tries to determine the absolute path by prepending
the current working directory.)

I'll ignore the autosplit.ix as it's not a problem.

The problem is that the foo.al files contain a #line directive like
#line 120 "../../lib/POSIX.pm (autosplit into ../../lib/auto/POSIX/errno.al)"
which perl dutifully accepts as the line number and filename.

So NYTProf see the filename as
"../../lib/POSIX.pm (autosplit into ../../lib/auto/POSIX/errno.al)"
It's smart enough to strip off the " (autosplit..." suffix, but that's all.
So we're left with a relative filename that bears little relation to reality.

What we really want to happen is for get_file_id() in NYTProf.xs to
return the same fid for both these strings:
"/usr/lib/perl5/5.8.8/i386-linux-thread-multi/POSIX.pm"
"../../lib/POSIX.pm (autosplit into ../../lib/auto/POSIX/errno.al)"

A slight complication is that get_file_id needs to be _very_ fast.
That means we need both strings to be in the hash table, so when
an autosplit entry is added we need to detect that and search for the
corresponding 'parent' fid and then assign the same fid to the autosplit
entry.

Here's a sketch of the steps I'd take (possibly wrong and probably
incomplete :-) ...

[... deleted ...]

I did write it out but then I figured it would be easier to do the
structural change myself and leave the more, er, interesting parts to
you :)

See r694

Now all you need to do is write find_autosplit_parent() :-)

That needs to iterate over the hash, like write_cached_fids() does,
and find an entry with a .key that 'matches' the file_name argument.
So for the example above, "../../lib/POSIX.pm (autosplit ...."
should match the "..../i386-linux-thread-multi/POSIX.pm" entry.
Clearly ignoring the directory portion is essential. I'd also
suggest not exiting the loop early, so you get the most recent
fid that matches, as that's most likely to be the right one.

I guess you can decide between yourselves who writes the code and who
writes the tests. Or have a race... both do both and I'll pick the
winner :)

Tim.

I have gotten a checkout, opened a ticket, and verified the bug on 3
different platforms. I didn't actually do anything about the bug yet,
though. =-)

If you can wait one more day, I will see if I can craft a patch by the
end of the day.