Pretty painful restarting the server

[Scott McIntyre]

It might take a wee bit more time to fix. I ran into some complications (of
course).

1) We can't set a flag in the signal handler and then test for it in our
main loop. Our main loop waits for network traffic, so the test wouldn't be
run constantly like it would need to. I think I might have to schedule a
task just to check this flag. Not very elegant.

2) MLDBM prefers to use the DB_File module as the underlying database
because it has an unlimited block size. My Perl 5 Pocket Reference says
that's one of the five DB modules included in the list of Standard Modules.
ActivePerl under Windows only feels the need to include one of those five --
and DB_File isn't the one. And of course they're all written in C, so you
can't just copy them into our CPAN dir. I have to see whether it even works
with the one available one. But it does work great under Linux. :)

Scott

----- Original Message -----
From: "dean blackketter" <dean-qV/***@public.gmane.org>
To: <slimp3-dev-***@public.gmane.org>
Sent: Sunday, February 03, 2002 12:20 PM
Subject: Re: [slimp3-dev] Pretty painful restarting the server

Scott,
Can we expect an updated patch from you? :-)
-dean

Yeah, that was me being lazy and ignoring the section of the docs that

says

nothing is re-entrant. Forgot about that. Also, I only put in a handler
for SIGINT, and it really should have one for die too.
Scott
----- Original Message -----
Sent: Sunday, February 03, 2002 4:00 AM
Subject: Re: [slimp3-dev] Pretty painful restarting the server

2) Installs a signal handler to cleanup the database properly when

you

kill the server.

FYI - signals are unreliable under perl.

This deserves a little better explanation.
The problem is that on some platforms, perl will not catch subsequent
signals when it's already in a signal handler, and those signals will

have

their default effect. Eg if we have a long sub for handling a SIGKILL,

a

second Control-C while we're in the handler would cause us to exit.
So our kill handler should just set a flag and return. Then we'd catch

the

flag in the main loop and call the shutdown procedure from there.

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Do you need to encrypt all your online transactions? Find
the perfect solution in this FREE Guide from VeriSign.
http://us.click.yahoo.com/vCuuSA/UdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

Scott,

Can we expect an updated patch from you? :-)

-dean

Yeah, that was me being lazy and ignoring the section of the docs that says
nothing is re-entrant. Forgot about that. Also, I only put in a handler
for SIGINT, and it really should have one for die too.
Scott
----- Original Message -----
Sent: Sunday, February 03, 2002 4:00 AM
Subject: Re: [slimp3-dev] Pretty painful restarting the server

2) Installs a signal handler to cleanup the database properly when you

kill the server.

FYI - signals are unreliable under perl.

This deserves a little better explanation.
The problem is that on some platforms, perl will not catch subsequent
signals when it's already in a signal handler, and those signals will have
their default effect. Eg if we have a long sub for handling a SIGKILL, a
second Control-C while we're in the handler would cause us to exit.
So our kill handler should just set a flag and return. Then we'd catch the
flag in the main loop and call the shutdown procedure from there.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Secure all your Web servers now - with a proven 5-part
strategy. The FREE Server Security Guide shows you how.
http://us.click.yahoo.com/uCuuSA/VdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Scott McIntyre]

Yeah, that was me being lazy and ignoring the section of the docs that says
nothing is re-entrant. Forgot about that. Also, I only put in a handler
for SIGINT, and it really should have one for die too.

Scott

----- Original Message -----
From: "Sean Adams" <sadams-C+JAmueIyFXt/WbtSZLAmwC/***@public.gmane.org>
To: <slimp3-dev-***@public.gmane.org>
Sent: Sunday, February 03, 2002 4:00 AM
Subject: Re: [slimp3-dev] Pretty painful restarting the server

2) Installs a signal handler to cleanup the database properly when you

kill the server.

FYI - signals are unreliable under perl.

This deserves a little better explanation.
The problem is that on some platforms, perl will not catch subsequent
signals when it's already in a signal handler, and those signals will have
their default effect. Eg if we have a long sub for handling a SIGKILL, a
second Control-C while we're in the handler would cause us to exit.
So our kill handler should just set a flag and return. Then we'd catch the
flag in the main loop and call the shutdown procedure from there.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Tiny Wireless Camera under $80!
Order Now! FREE VCR Commander!
Click Here - Only 1 Day Left!
http://us.click.yahoo.com/nuyOHD/7.PDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>Re: [slimp3-dev] Pretty painful restarting the
server</title></head><body>
<div>Scott,</div>
<div><br></div>
<div>Can we expect an updated patch from you?&nbsp; :-)</div>
<div><br></div>
<div>-dean</div>
<div><br></div>
<blockquote type="cite" cite>Yeah, that was me being lazy and ignoring
the section of the docs that says<br>
nothing is re-entrant.&nbsp; Forgot about that.&nbsp; Also, I only put
in a handler<br>
for SIGINT, and it really should have one for die too.<br>
<br>
Scott<br>
<br>
----- Original Message -----<br>
From: &quot;Sean Adams&quot; &lt;sadams-C+JAmueIyFXt/WbtSZLAmwC/***@public.gmane.org&gt;<br>
To: &lt;slimp3-dev-***@public.gmane.org&gt;<br>
Sent: Sunday, February 03, 2002 4:00 AM<br>
Subject: Re: [slimp3-dev] Pretty painful restarting the server<br>
<br>
<br>
&gt;<br>
&gt; &gt; &gt; 2) Installs a signal handler to cleanup the database
properly when you<br>
kill the server.<br>
&gt; &gt;<br>
&gt; &gt; FYI - signals are unreliable under perl.<br>
&gt;<br>
&gt; This deserves a little better explanation.<br>
&gt;<br>
&gt; The problem is that on some platforms, perl will not catch
subsequent<br>
&gt; signals when it's already in a signal handler, and those signals
will have<br>
&gt; their default effect. Eg if we have a long sub for handling a
SIGKILL, a<br>
&gt; second Control-C while we're in the handler would cause us to
exit.<br>
&gt;<br>
&gt; So our kill handler should just set a flag and return. Then we'd
catch the<br>
&gt; flag in the main loop and call the shutdown procedure from
there.<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; To unsubscribe from this group, send an email to:<br>
&gt; slimp3-dev-unsubscribe-***@public.gmane.org<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/<br>
&gt;<br>
&gt;<br>
<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Tiny Wireless Camera under $80!<br>
Order Now! FREE VCR Commander!<br>
Click Here - Only 1 Day Left!<br>
http://us.click.yahoo.com/nuyOHD/7.PDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>

[dean blackketter]

GREAT!

I'll give this a run and check it in.

Nice work Scott.

-dean

The attached patch expands on and includes what I sent to the list earlier.
It keeps the mp3tag cache on a DB file without a performance hit, using the
two level cache I proposed earlier. It also regenerates the artist cache at
startup from the database, which only takes a second or two (on a 300 Mhz
machine).
The net result is that you can kill the server and restart it and it doesn't
suck. Even better is if you explicitly scan your entire library, by doing a
Play on the top level "Music Library" choice (which takes about 15 seconds
for my 3758 files, and generates a 1.3 MB database file). Then you have a
complete artists menu, and every directory is pre-cached. Yay!
Must go to sleep now. :)
Scott
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
Content-Type: application/octet-stream;
name="db.patch"
Content-Disposition: attachment;
filename="db.patch"
Attachment converted: firebook:db.patch (????/----) (000AB460)

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Secure all your Web servers now - with a proven 5-part
strategy. The FREE Server Security Guide shows you how.
http://us.click.yahoo.com/uCuuSA/VdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>Re: [slimp3-dev] Pretty painful restarting the
server</title></head><body>
<div>GREAT! </div>
<div><br></div>
<div>I'll give this a run and check it in.</div>
<div><br></div>
<div>Nice work Scott.</div>
<div><br></div>
<div>-dean</div>
<div><br></div>
<blockquote type="cite" cite>The attached patch expands on and
includes what I sent to the list earlier.<br>
It keeps the mp3tag cache on a DB file without a performance hit,
using the<br>
two level cache I proposed earlier.&nbsp; It also regenerates the
artist cache at<br>
startup from the database, which only takes a second or two (on a 300
Mhz<br>
machine).<br>
<br>
The net result is that you can kill the server and restart it and it
doesn't<br>
suck.&nbsp; Even better is if you explicitly scan your entire library,
by doing a<br>
Play on the top level &quot;Music Library&quot; choice (which takes
about 15 seconds<br>
for my 3758 files, and generates a 1.3 MB database file).&nbsp; Then
you have a<br>
complete artists menu, and every directory is pre-cached. Yay!<br>
<br>
Must go to sleep now. :)<br>
<br>
Scott<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Pinpoint the right security solution for your company - FREE<br>
Guide from industry leader VeriSign gives you all the facts.<br>
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ <br>
<br>
<br>
Content-Type: application/octet-stream;<br>
<x-tab>&nbsp; </x-tab>name=&quot;db.patch&quot;<br>
Content-Disposition: attachment;</blockquote>
<blockquote type="cite"
cite><x-tab>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</x-tab>filename=&quot;db.patch&quot;</blockquote>
<blockquote type="cite" cite><br></blockquote>
<blockquote type="cite" cite>Attachment converted: firebook:db.patch
(????/----) (000AB460)</blockquote>
<div><br></div>
</body>
</html>

[Sean Adams]

2) Installs a signal handler to cleanup the database properly when you kill the server.

FYI - signals are unreliable under perl.

This deserves a little better explanation.

The problem is that on some platforms, perl will not catch subsequent
signals when it's already in a signal handler, and those signals will have
their default effect. Eg if we have a long sub for handling a SIGKILL, a
second Control-C while we're in the handler would cause us to exit.

So our kill handler should just set a flag and return. Then we'd catch the
flag in the main loop and call the shutdown procedure from there.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Secure all your Web servers now - with a proven 5-part
strategy. The FREE Server Security Guide shows you how.
http://us.click.yahoo.com/iWSNbC/VdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Daniel Sully]

Once upon a time Scott McIntyre shaped the electrons to say...

2) Installs a signal handler to cleanup the database properly when you kill the server.

FYI - signals are unreliable under perl.

-D
--
estupidos pantalones

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/pCuuSA/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Scott McIntyre]

The attached patch expands on and includes what I sent to the list earlier.
It keeps the mp3tag cache on a DB file without a performance hit, using the
two level cache I proposed earlier. It also regenerates the artist cache at
startup from the database, which only takes a second or two (on a 300 Mhz
machine).

The net result is that you can kill the server and restart it and it doesn't
suck. Even better is if you explicitly scan your entire library, by doing a
Play on the top level "Music Library" choice (which takes about 15 seconds
for my 3758 files, and generates a 1.3 MB database file). Then you have a
complete artists menu, and every directory is pre-cached. Yay!

Must go to sleep now. :)

Scott

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Scott McIntyre]

I've been playing around with this a little. I've attached an example patch
to do it "the perl way".

What it does is:
1) Associates the mp3tagCache hash with a MLDBM file using a tie. You have
to use the MLDBM package if your structures are hashes of hashes.
2) Installs a signal handler to cleanup the database properly when you kill
the server.

This is the dead simplest way to do it. Only 15 lines of code. As I coded
it, it has the rather large disadvantage that every hash reference is now a
database reference, which causes a performance hit in just normal browsing.
I think that could be resolved by implementing a 2 level cache, where a read
would look like this:

check memory cache
if found, return value
if not, check database cache
if found, return value and update memory cache
if not, generate value, update database cache and memory cache

Which would make every read after the first be as fast as the current
system.

Just some stuff to ponder...

Scott

Index: server.pl
===================================================================
RCS file: /cvsroot/slimp3/slimp3/server/server.pl,v
retrieving revision 1.85
diff -u -p -B -r1.85 server.pl
--- server.pl 2002/02/02 04:59:40 1.85
+++ server.pl 2002/02/03 06:16:24
@@ -64,6 +64,7 @@ use SliMP3::Control;
use SliMP3::Display;
use SliMP3::HTTP;
use SliMP3::IR;
+use SliMP3::Misc;
use SliMP3::OSDetect;
use SliMP3::Playlist;
use SliMP3::Prefs;
@@ -229,6 +230,13 @@ sub main {
SliMP3::Discovery::sayHello($udpsock, $addr);
}
}
+
+ SliMP3::Misc::initCache();
+
+ $SIG{INT} = sub {
+ SliMP3::Misc::stopCache();
+ exit;
+ };

# start scanning based on a timer...
# Currently, it's set to one item (directory or song) scanned per second.

Index: Misc.pm
===================================================================
RCS file: /cvsroot/slimp3/slimp3/server/lib/SliMP3/Misc.pm,v
retrieving revision 1.32
diff -r1.32 Misc.pm
9a10

use MLDBM qw(DB_File Storable);

19a21,30

sub initCache {
tie (%mp3tagCache, 'MLDBM', 'mp3tag.db')
or die "Can't tie to mp3.db: $!";
}
sub stopCache {
print "Shutting down mp3 tag database.\n";
untie (%mp3tagCache);
}

557c568
< updateCache($curPath);
---

mp3tag($curPath);

----- Original Message -----
From: "dean blackketter" <dean-qV/***@public.gmane.org>
To: <slimp3-dev-***@public.gmane.org>
Sent: Saturday, February 02, 2002 11:45 PM
Subject: Re: [slimp3-dev] Pretty painful restarting the server

Sean,
As an example, could you take one of our existing caches and move it
to a DBM file? I started to look at this, but got stuck trying to
figure out how to make it work without keeping the file open all the
time. If the server dies or gets killed, I'm afraid that the DBM
file might get corrupted. If we open and close it on every access,
then I'm afraid that we'll have a serious performance penalty.
Also, since we are building complex data structures (hashes of
hashes), I'd need some help figuring out how do do this without
building out a multitude of separate DBM files...
-dean

Agreed - I feel very strongly about using perl's built-in dbm capability
for this.
- fast random access
- no startup/shutdown wait
- caching built-in to minimze fs access
- easy to add/remove fields
- very easy to migrate our existing design
If you want to be able to view/edit the data for debugging, all it takes
is a four-liner scripts to convert to/from the format described below.

Please use a pack()'ed file or BerkeleyDB database to store this type

of

information - parsing and writing a sequential text file may be
"simple", but
won't scale. Something hashed based for key access is good.
Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across

pseudo-restarts,

but don't we need an efficient way to keep the indexes and fs

cache across

real restarts, anyway?

For saving the cache, we could just dump everything to a giant text

file,

each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our

Way.mp3]

FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with

a VB

application, wrote code to traverse a directory structure and get

all .mp3

files (takes a couple of seconds) then get the ID3 tags for them

all (takes

less than 4 minutes) then dump all the information to a file like

that

(which it saves in a couple of seconds). Over 4,000 tracks. Can load

the

whole file into a VB array in a couple of seconds too, so I'm

sure Perl can

do it just as fast.

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Sean Adams]

Sure, I can have a go at it. It'll be later next week though - gotta get
the new Scan.pm finished first.

Sean,
As an example, could you take one of our existing caches and move it
to a DBM file? I started to look at this, but got stuck trying to
figure out how to make it work without keeping the file open all the
time. If the server dies or gets killed, I'm afraid that the DBM
file might get corrupted. If we open and close it on every access,
then I'm afraid that we'll have a serious performance penalty.
Also, since we are building complex data structures (hashes of
hashes), I'd need some help figuring out how do do this without
building out a multitude of separate DBM files...
-dean

Agreed - I feel very strongly about using perl's built-in dbm capability
for this.
- fast random access
- no startup/shutdown wait
- caching built-in to minimze fs access
- easy to add/remove fields
- very easy to migrate our existing design
If you want to be able to view/edit the data for debugging, all it takes
is a four-liner scripts to convert to/from the format described below.

Please use a pack()'ed file or BerkeleyDB database to store this type of
information - parsing and writing a sequential text file may be
"simple", but
won't scale. Something hashed based for key access is good.
Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across

pseudo-restarts,

but don't we need an efficient way to keep the indexes and fs

cache across

real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them

all (takes

less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm

sure Perl can

do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

-D
--
C program run, C program crash, C programmer cry.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

Well, packing and or compressing is good to speed up I/O, but if it's
going to be read in once at startup time and loaded into memory, then
there's no reason not to use a flat file.

Of course, once it's in memory, a hash (or whatever appropriate data
structure) will be used to access it for a zippy user interface.

The file would have to be opened for append when a new item gets
added to the cache, but this would only happen during scanning or
when a new music file si discovered...

-dean

Please use a pack()'ed file or BerkeleyDB database to store this type of
information - parsing and writing a sequential text file may be "simple", but
won't scale. Something hashed based for key access is good.
Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them all (takes
less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm sure Perl can
do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

-D
--
C program run, C program crash, C programmer cry.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/pCuuSA/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

Sean,

As an example, could you take one of our existing caches and move it
to a DBM file? I started to look at this, but got stuck trying to
figure out how to make it work without keeping the file open all the
time. If the server dies or gets killed, I'm afraid that the DBM
file might get corrupted. If we open and close it on every access,
then I'm afraid that we'll have a serious performance penalty.

Also, since we are building complex data structures (hashes of
hashes), I'd need some help figuring out how do do this without
building out a multitude of separate DBM files...

-dean

Agreed - I feel very strongly about using perl's built-in dbm capability
for this.
- fast random access
- no startup/shutdown wait
- caching built-in to minimze fs access
- easy to add/remove fields
- very easy to migrate our existing design
If you want to be able to view/edit the data for debugging, all it takes
is a four-liner scripts to convert to/from the format described below.

Please use a pack()'ed file or BerkeleyDB database to store this type of
information - parsing and writing a sequential text file may be
"simple", but
won't scale. Something hashed based for key access is good.
Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across

pseudo-restarts,

but don't we need an efficient way to keep the indexes and fs

cache across

real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them

all (takes

less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm

sure Perl can

do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

-D
--
C program run, C program crash, C programmer cry.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
When building an e-commerce site, you want to start with a
secure foundation. Learn how with VeriSign's FREE Guide.
http://us.click.yahoo.com/kWSNbC/XdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>Re: [slimp3-dev] Pretty painful restarting the
server</title></head><body>
<div>Well, packing and or compressing is good to speed up I/O, but if
it's going to be read in once at startup time and loaded into memory,
then there's no reason not to use a flat file.</div>
<div><br></div>
<div>Of course, once it's in memory, a hash (or whatever appropriate
data structure) will be used to access it for a zippy user interface.&nbsp;
</div>
<div><br></div>
<div>The file would have to be opened for append when a new item gets
added to the cache, but this would only happen during scanning or when
a new music file si discovered...</div>
<div><br></div>
<div>-dean</div>
<div><br></div>
<blockquote type="cite" cite>Please use a pack()'ed file or BerkeleyDB
database to store this type of<br>
information - parsing and writing a sequential text file may be
&quot;simple&quot;, but<br>
won't scale. Something hashed based for key access is good.<br>
<br>
Once upon a time Richard Smith shaped the electrons to say...<br>
<br>
&gt; &gt; This is interesting - would let us save the cache across
pseudo-restarts,<br>
&gt; &gt; but don't we need an efficient way to keep the indexes and
fs cache across<br>
&gt; &gt; real restarts, anyway?<br>
&gt; <br>
&gt; For saving the cache, we could just dump everything to a giant
text file,<br>
&gt; each bit of information on a new line, including all the ID3
info...<br>
&gt; <br>
&gt; [c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our
Way.mp3]<br>
&gt; FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\<br>
&gt; FileName=13 - Find Our Way.mp3<br>
&gt; Album=Tone Soul Evolution<br>
&gt; Artist=Apples In Stereo<br>
&gt; Genre=N/A<br>
&gt; Length=147<br>
&gt; Song=Find Our Way<br>
&gt; Track=13<br>
&gt; Year=1997<br>
&gt; <br>
&gt; Then just read it all back in again on server load. I did this
with a VB<br>
&gt; application, wrote code to traverse a directory structure and get
all .mp3<br>
&gt; files (takes a couple of seconds) then get the ID3 tags for them
all (takes<br>
&gt; less than 4 minutes) then dump all the information to a file like
that<br>
&gt; (which it saves in a couple of seconds). Over 4,000 tracks. Can
load the<br>
&gt; whole file into a VB array in a couple of seconds too, so I'm
sure Perl can<br>
&gt; do it just as fast.<br>
&gt; <br>
&gt; <br>
&gt; <br>
&gt; <br>
&gt; Richard Smith<br>
&gt; richard-+***@public.gmane.org<br>
&gt; <br>
&gt; - Stay up to date with all your favourite news sources...<br>
&gt; - http://www.bearkey.com/alerts/<br>
&gt; - BBC, CNN, Slashdot, Wired, CNET and more.<br>
&gt; <br>
&gt; <br>
&gt; <br>
&gt; To unsubscribe from this group, send an email to:<br>
&gt; slimp3-dev-unsubscribe-***@public.gmane.org<br>
&gt; <br>
&gt;&nbsp; <br>
&gt; <br>
&gt; Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ <br>
&gt; <br>
&gt; <br>
<br>
-D<br>
--<br>
C program run, C program crash, C programmer cry.<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Do you need to encrypt all your online transactions? Find<br>
the perfect solution in this FREE Guide from VeriSign.<br>
http://us.click.yahoo.com/jWSNbC/UdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>Re: [slimp3-dev] Pretty painful restarting the
server</title></head><body>
<div>Sean,</div>
<div><br></div>
<div>As an example, could you take one of our existing caches and move
it to a DBM file?&nbsp; I started to look at this, but got stuck
trying to figure out how to make it work without keeping the file open
all the time.&nbsp; If the server dies or gets killed, I'm afraid that
the DBM file might get corrupted.&nbsp; If we open and close it on
every access, then I'm afraid that we'll have a serious performance
penalty.&nbsp;&nbsp; </div>
<div><br></div>
<div>Also, since we are building complex data structures (hashes of
hashes), I'd need some help figuring out how do do this without
building out a multitude of separate DBM files...</div>
<div><br></div>
<div>-dean</div>
<div><br></div>
<div><br></div>
<blockquote type="cite" cite>Agreed - I feel very strongly about using
perl's built-in dbm capability<br>
for this.<br>
<br>
- fast random access<br>
- no startup/shutdown wait<br>
- caching built-in to minimze fs access<br>
- easy to add/remove fields<br>
- very easy to migrate our existing design<br>
<br>
If you want to be able to view/edit the data for debugging, all it
takes<br>
is a four-liner scripts to convert to/from the format described
below.<br>
<br>
On Sat, 2 Feb 2002, Daniel Sully wrote:<br>
<br>
&gt; Please use a pack()'ed file or BerkeleyDB database to store this
type of<br>
&gt; information - parsing and writing a sequential text file may be
&quot;simple&quot;, but<br>
&gt; won't scale. Something hashed based for key access is good.<br>
&gt;<br>
&gt; Once upon a time Richard Smith shaped the electrons to say...<br>
&gt;<br>
&gt; &gt; &gt; This is interesting - would let us save the cache
across pseudo-restarts,<br>
&gt; &gt; &gt; but don't we need an efficient way to keep the indexes
and fs cache across<br>
&gt; &gt; &gt; real restarts, anyway?<br>
&gt; &gt;<br>
&gt; &gt; For saving the cache, we could just dump everything to a
giant text file,<br>
&gt; &gt; each bit of information on a new line, including all the ID3
info...<br>
&gt; &gt;<br>
&gt; &gt; [c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 -
Find Our Way.mp3]<br>
&gt; &gt; FilePath=c:\soundmusic\Apples in Stereo\Tone Soul
Evolution\<br>
&gt; &gt; FileName=13 - Find Our Way.mp3<br>
&gt; &gt; Album=Tone Soul Evolution<br>
&gt; &gt; Artist=Apples In Stereo<br>
&gt; &gt; Genre=N/A<br>
&gt; &gt; Length=147<br>
&gt; &gt; Song=Find Our Way<br>
&gt; &gt; Track=13<br>
&gt; &gt; Year=1997<br>
&gt; &gt;<br>
&gt; &gt; Then just read it all back in again on server load. I did
this with a VB<br>
&gt; &gt; application, wrote code to traverse a directory structure
and get all .mp3<br>
&gt; &gt; files (takes a couple of seconds) then get the ID3 tags for
them all (takes<br>
&gt; &gt; less than 4 minutes) then dump all the information to a file
like that<br>
&gt; &gt; (which it saves in a couple of seconds). Over 4,000 tracks.
Can load the<br>
&gt; &gt; whole file into a VB array in a couple of seconds too, so
I'm sure Perl can<br>
&gt; &gt; do it just as fast.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Richard Smith<br>
&gt; &gt; richard-+***@public.gmane.org<br>
&gt; &gt;<br>
&gt; &gt; - Stay up to date with all your favourite news
sources...<br>
&gt; &gt; - http://www.bearkey.com/alerts/<br>
&gt; &gt; - BBC, CNN, Slashdot, Wired, CNET and more.<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; To unsubscribe from this group, send an email to:<br>
&gt; &gt; slimp3-dev-unsubscribe-***@public.gmane.org<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt; &gt; Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/<br>
&gt; &gt;<br>
&gt; &gt;<br>
&gt;<br>
&gt; -D<br>
&gt; --<br>
&gt; C program run, C program crash, C programmer cry.<br>
&gt;<br>
&gt;<br>
&gt; To unsubscribe from this group, send an email to:<br>
&gt; slimp3-dev-unsubscribe-***@public.gmane.org<br>
&gt;<br>
&gt;<br>
&gt;<br>
&gt; Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/<br>
&gt;<br>
&gt;<br>
&gt;<br>
<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Pinpoint the right security solution for your company - FREE<br>
Guide from industry leader VeriSign gives you all the facts.<br>
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>

[Sean Adams]

Agreed - I feel very strongly about using perl's built-in dbm capability
for this.

- fast random access
- no startup/shutdown wait
- caching built-in to minimze fs access
- easy to add/remove fields
- very easy to migrate our existing design

If you want to be able to view/edit the data for debugging, all it takes
is a four-liner scripts to convert to/from the format described below.

Please use a pack()'ed file or BerkeleyDB database to store this type of
information - parsing and writing a sequential text file may be "simple", but
won't scale. Something hashed based for key access is good.
Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them all (takes
less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm sure Perl can
do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

-D
--
C program run, C program crash, C programmer cry.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Daniel Sully]

Please use a pack()'ed file or BerkeleyDB database to store this type of
information - parsing and writing a sequential text file may be "simple", but
won't scale. Something hashed based for key access is good.

Once upon a time Richard Smith shaped the electrons to say...

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them all (takes
less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm sure Perl can
do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

-D
--
C program run, C program crash, C programmer cry.

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Do you need to encrypt all your online transactions? Find
the perfect solution in this FREE Guide from VeriSign.
http://us.click.yahoo.com/jWSNbC/UdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

This is probably the right thing to do in the long run.

$code = "$x++";

Need to escape that $ sign...

Ooops.

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997
Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them all (takes
less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm sure Perl can
do it just as fast.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Richard Smith]

$code = "$x++";

Need to escape that $ sign...

Ooops.

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

For saving the cache, we could just dump everything to a giant text file,
each bit of information on a new line, including all the ID3 info...

[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our Way.mp3]
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\
FileName=13 - Find Our Way.mp3
Album=Tone Soul Evolution
Artist=Apples In Stereo
Genre=N/A
Length=147
Song=Find Our Way
Track=13
Year=1997

Then just read it all back in again on server load. I did this with a VB
application, wrote code to traverse a directory structure and get all .mp3
files (takes a couple of seconds) then get the ID3 tags for them all (takes
less than 4 minutes) then dump all the information to a file like that
(which it saves in a couple of seconds). Over 4,000 tracks. Can load the
whole file into a VB array in a couple of seconds too, so I'm sure Perl can
do it just as fast.




Richard Smith
richard-+***@public.gmane.org

- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Do you need to encrypt all your online transactions? Find
the perfect solution in this FREE Guide from VeriSign.
http://us.click.yahoo.com/jWSNbC/UdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>RE: [slimp3-dev] Pretty painful restarting the
server</title></head><body> <div>This is probably the right thing to do in the long run.</div> <div><br></div> <div><br></div> <blockquote type="cite" cite>&gt; &gt; $code = &quot;$x++&quot;;<br>
&gt;<br>
&gt; Need to escape that $ sign...<br>
<br>
Ooops.<br>
<br>
&gt; This is interesting - would let us save the cache across
pseudo-restarts,<br>
&gt; but don't we need an efficient way to keep the indexes and fs
cache across<br>
&gt; real restarts, anyway?<br>
<br>
For saving the cache, we could just dump everything to a giant text
file,<br>
each bit of information on a new line, including all the ID3
info...<br>
<br>
[c:\soundmusic\Apples in Stereo\Tone Soul Evolution\13 - Find Our
Way.mp3]<br>
FilePath=c:\soundmusic\Apples in Stereo\Tone Soul Evolution\<br>
FileName=13 - Find Our Way.mp3<br>
Album=Tone Soul Evolution<br>
Artist=Apples In Stereo<br>
Genre=N/A<br>
Length=147<br>
Song=Find Our Way<br>
Track=13<br>
Year=1997<br>
<br>
Then just read it all back in again on server load. I did this with a
VB<br>
application, wrote code to traverse a directory structure and get all
.mp3<br>
files (takes a couple of seconds) then get the ID3 tags for them all
(takes<br>
less than 4 minutes) then dump all the information to a file like
that<br>
(which it saves in a couple of seconds). Over 4,000 tracks. Can load
the<br>
whole file into a VB array in a couple of seconds too, so I'm sure
Perl can<br>
do it just as fast.<br>
<br>
<br>
<br>
<br>
Richard Smith<br>
richard-+***@public.gmane.org<br>
<br>
- Stay up to date with all your favourite news sources...<br>
- http://www.bearkey.com/alerts/<br>
- BBC, CNN, Slashdot, Wired, CNET and more.<br>
<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Do you need to encrypt all your online transactions? Find<br>
the perfect solution in this FREE Guide from VeriSign.<br>
http://us.click.yahoo.com/jWSNbC/UdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>

[Richard Smith]

I'd love to (I'd love to help more) but I took on the job of writing the
docs and I still haven't found time to do that yet. :(

Real-Life Work (lost a developer from our Perl team, which means more work
than ever) combined with running chatbear.com and the service in my
signature barely leaves me enough time to actually eat and sleep. So at the
moment I think it'd be a bad idea if I said I could manage this, however
much I'd like to.

A real hack might simply be to write a script that loaded in server.pl then
eval'd that. Not sure whether that would work though...


Richard Smith
richard-+***@public.gmane.org

- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.

-----Original Message-----
Sent: 03 February 2002 00:52
Subject: RE: [slimp3-dev] Pretty painful restarting the server
Cool. Do you want to take this on?
-dean

1. Will this have any impact on the performance of the server?

No, Perl will execute the code with eval the same way as it would execute
any normal code. A good example of this is mod_perl, which

effectively wraps

all your code in an eval with no speed-hit.

2. It seems to me that if any of the caching data structures change,
then the server would have to be completely restarted, right?

Interesting. The way Net::IRC works is event driven. Each possible IRC
server action is tied to a sub, which is called by an event

handler. It's a

very clean way to handle the server code as it allowed me to

take each of my

event handlers, put them all in different files and basically

make the event

handler sub just the eval of that file. I could then change any of those
files and just reload it and the next time the event fired, the

code in it

was used. I never had any problems with data structures being wrong, or
variables having the wrong values, and I did store some persistent
information. As long as the slimp3 code could handle the data structure
maybe missing some parts, and fill in what wasn't there, then

you shouldn't

have any issues. But then, how often is the data structure going

to change

anyway? You're more likely to change the functions that use the data.

2. I understand why we developers need to restart the server
frequently. Why would non-developer customers need to?

Shouldn't this be question 3? ;)
Putting everything in an eval makes it easier for developers (natch) but
customers too. You could build in a simple auto-updater that checked for
latest versions, downloaded them and installed them without the

server ever

stopping.
Eval example for people not quite Perl sure...
This should work...
$x = 1;
$code = "$x++";
eval { $code };
print $x;

2

Run eval { $code }; again and $x will equal 3 and so on. $code can be any
valid Perl code. A whole script.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Sean Adams]

$code = "$x++";

Need to escape that $ sign...

This is interesting - would let us save the cache across pseudo-restarts,
but don't we need an efficient way to keep the indexes and fs cache across
real restarts, anyway?

No, Perl will execute the code with eval the same way as it would
execute any normal code.

True, sorta. It's just like starting up a script. You get the compile
penalty when you go into the eval, but once you're IN the eval, it's like
normal, I agree. So you *do* have to limit your use of eval in general. In
this case it would be okay.

Sean


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Secure all your Web servers now - with a proven 5-part
strategy. The FREE Server Security Guide shows you how.
http://us.click.yahoo.com/iWSNbC/VdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

Cool. Do you want to take this on?

-dean

1. Will this have any impact on the performance of the server?

No, Perl will execute the code with eval the same way as it would execute
any normal code. A good example of this is mod_perl, which effectively wraps
all your code in an eval with no speed-hit.

2. It seems to me that if any of the caching data structures change,
then the server would have to be completely restarted, right?

Interesting. The way Net::IRC works is event driven. Each possible IRC
server action is tied to a sub, which is called by an event handler. It's a
very clean way to handle the server code as it allowed me to take each of my
event handlers, put them all in different files and basically make the event
handler sub just the eval of that file. I could then change any of those
files and just reload it and the next time the event fired, the code in it
was used. I never had any problems with data structures being wrong, or
variables having the wrong values, and I did store some persistent
information. As long as the slimp3 code could handle the data structure
maybe missing some parts, and fill in what wasn't there, then you shouldn't
have any issues. But then, how often is the data structure going to change
anyway? You're more likely to change the functions that use the data.

2. I understand why we developers need to restart the server
frequently. Why would non-developer customers need to?

Shouldn't this be question 3? ;)
Putting everything in an eval makes it easier for developers (natch) but
customers too. You could build in a simple auto-updater that checked for
latest versions, downloaded them and installed them without the server ever
stopping.
Eval example for people not quite Perl sure...
This should work...
$x = 1;
$code = "$x++";
eval { $code };
print $x;

2

Run eval { $code }; again and $x will equal 3 and so on. $code can be any
valid Perl code. A whole script.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Tiny Wireless Camera under $80!
Order Now! FREE VCR Commander!
Click Here - Only 1 Day Left!
http://us.click.yahoo.com/nuyOHD/7.PDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Richard Smith]

1. Will this have any impact on the performance of the server?

No, Perl will execute the code with eval the same way as it would execute
any normal code. A good example of this is mod_perl, which effectively wraps
all your code in an eval with no speed-hit.

2. It seems to me that if any of the caching data structures change,
then the server would have to be completely restarted, right?

Interesting. The way Net::IRC works is event driven. Each possible IRC
server action is tied to a sub, which is called by an event handler. It's a
very clean way to handle the server code as it allowed me to take each of my
event handlers, put them all in different files and basically make the event
handler sub just the eval of that file. I could then change any of those
files and just reload it and the next time the event fired, the code in it
was used. I never had any problems with data structures being wrong, or
variables having the wrong values, and I did store some persistent
information. As long as the slimp3 code could handle the data structure
maybe missing some parts, and fill in what wasn't there, then you shouldn't
have any issues. But then, how often is the data structure going to change
anyway? You're more likely to change the functions that use the data.

2. I understand why we developers need to restart the server
frequently. Why would non-developer customers need to?

Shouldn't this be question 3? ;)

Putting everything in an eval makes it easier for developers (natch) but
customers too. You could build in a simple auto-updater that checked for
latest versions, downloaded them and installed them without the server ever
stopping.


Eval example for people not quite Perl sure...

This should work...

$x = 1;
$code = "$x++";

eval { $code };

print $x;

2

Run eval { $code }; again and $x will equal 3 and so on. $code can be any
valid Perl code. A whole script.




Richard Smith
richard-+***@public.gmane.org

- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.


------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>RE: [slimp3-dev] Pretty painful restarting the
server</title></head><body> <div>Cool.&nbsp; Do you want to take this on?</div> <div><br></div> <div>-dean</div> <div><br></div> <blockquote type="cite" cite>&gt; 1. Will this have any impact on the
performance of the server?<br>
<br>
No, Perl will execute the code with eval the same way as it would
execute<br>
any normal code. A good example of this is mod_perl, which effectively
wraps<br>
all your code in an eval with no speed-hit.<br>
<br>
&gt; 2. It seems to me that if any of the caching data structures
change,<br>
&gt; then the server would have to be completely restarted,
right?</blockquote>
<blockquote type="cite" cite><br>
Interesting. The way Net::IRC works is event driven. Each possible
IRC<br>
server action is tied to a sub, which is called by an event handler.
It's a<br>
very clean way to handle the server code as it allowed me to take each
of my<br>
event handlers, put them all in different files and basically make the
event<br>
handler sub just the eval of that file. I could then change any of
those<br>
files and just reload it and the next time the event fired, the code
in it<br>
was used. I never had any problems with data structures being wrong,
or<br>
variables having the wrong values, and I did store some persistent<br>
information. As long as the slimp3 code could handle the data
structure<br>
maybe missing some parts, and fill in what wasn't there, then you
shouldn't<br>
have any issues. But then, how often is the data structure going to
change<br>
anyway? You're more likely to change the functions that use the
data.<br>
<br>
&gt; 2. I understand why we developers need to restart the server<br>
&gt; frequently.&nbsp; Why would non-developer customers need
to?</blockquote>
<blockquote type="cite" cite><br>
Shouldn't this be question 3? ;)</blockquote>
<blockquote type="cite" cite><br>
Putting everything in an eval makes it easier for developers (natch)
but<br>
customers too. You could build in a simple auto-updater that checked
for<br>
latest versions, downloaded them and installed them without the server
ever<br>
stopping.<br>
</blockquote>
<blockquote type="cite" cite><br>
Eval example for people not quite Perl sure...<br>
<br>
This should work...<br>
<br>
$x = 1;<br>
$code = &quot;$x++&quot;;<br>
<br>
eval { $code };<br>
<br>
print $x;<br>
<br>
&gt; 2<br>
<br>
Run eval { $code }; again and $x will equal 3 and so on. $code can be
any<br>
valid Perl code. A whole script.<br>
</blockquote>
<blockquote type="cite" cite><br>
<br>
<br>
Richard Smith<br>
richard-+***@public.gmane.org<br>
<br>
- Stay up to date with all your favourite news sources...</blockquote>
<blockquote type="cite" cite>- http://www.bearkey.com/alerts/<br>
- BBC, CNN, Slashdot, Wired, CNET and more.<br>
<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Pinpoint the right security solution for your company - FREE<br>
Guide from industry leader VeriSign gives you all the facts.<br>
http://us.click.yahoo.com/lWSNbC/WdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>

[dean blackketter]

Richard,

This is a very interesting suggestion. Three questions:

1. Will this have any impact on the performance of the server?

2. It seems to me that if any of the caching data structures change,
then the server would have to be completely restarted, right?

2. I understand why we developers need to restart the server
frequently. Why would non-developer customers need to?

-dean

One way to stop the customer having to keep restarting the server is to have
the server itself (and the script you run to start it) be nothing more than
something that loads in all the code from each of our files into a variable
then does...
eval { $ourcode }
Then you can change the code in all the different parts of the server
whenever you want and then just reload everything into that variable when
you want to restart the server. Nothing gets stopped and restarted, no
pre-caches are lost, and the option to refresh the code can be something
selected from a menu on the slimp3 display.
It'll mean some restructuring of the current server to support this, but I
wrote an IRC bot with Net::IRC this way, storing each possible IRC event's
code in a different file and when I added new features I just told the bot
to reload his own code and that way he never had to disconnect from the
server.
Richard Smith
- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Do you need to encrypt all your online transactions? Find
the perfect solution in this FREE Guide from VeriSign.
http://us.click.yahoo.com/vCuuSA/UdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[Richard Smith]

One way to stop the customer having to keep restarting the server is to have
the server itself (and the script you run to start it) be nothing more than
something that loads in all the code from each of our files into a variable
then does...

eval { $ourcode }

Then you can change the code in all the different parts of the server
whenever you want and then just reload everything into that variable when
you want to restart the server. Nothing gets stopped and restarted, no
pre-caches are lost, and the option to refresh the code can be something
selected from a menu on the slimp3 display.

It'll mean some restructuring of the current server to support this, but I
wrote an IRC bot with Net::IRC this way, storing each possible IRC event's
code in a different file and when I added new features I just told the bot
to reload his own code and that way he never had to disconnect from the
server.



Richard Smith
richard-+***@public.gmane.org

- Stay up to date with all your favourite news sources...
- http://www.bearkey.com/alerts/
- BBC, CNN, Slashdot, Wired, CNET and more.




------------------------ Yahoo! Groups Sponsor ---------------------~-->
Sponsored by VeriSign - The Value of Trust
Pinpoint the right security solution for your company - FREE
Guide from industry leader VeriSign gives you all the facts.
http://us.click.yahoo.com/pCuuSA/WdiDAA/yigFAA/rIp0lB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
slimp3-dev-unsubscribe-***@public.gmane.org



Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/

[dean blackketter]

<!doctype html public "-//W3C//DTD W3 HTML//EN">
<head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>Re: [slimp3-dev] Pretty painful restarting the
server</title></head><body>
<div>Richard,</div>
<div><br></div>
<div>This is a very interesting suggestion.&nbsp;&nbsp; Three
questions:</div>
<div><br></div>
<div>1. Will this have any impact on the performance of the server?
</div>
<div><br></div>
<div>2. It seems to me that if any of the caching data structures
change, then the server would have to be completely restarted,
right?</div>
<div><br></div>
<div>2. I understand why we developers need to restart the server
frequently.&nbsp; Why would non-developer customers need to?</div>
<div><br></div>
<div>-dean</div>
<div><br></div>
<blockquote type="cite" cite>One way to stop the customer having to
keep restarting the server is to have<br>
the server itself (and the script you run to start it) be nothing more
than<br>
something that loads in all the code from each of our files into a
variable<br>
then does...<br>
<br>
eval { $ourcode }<br>
<br>
Then you can change the code in all the different parts of the
server<br>
whenever you want and then just reload everything into that variable
when<br>
you want to restart the server. Nothing gets stopped and restarted,
no<br>
pre-caches are lost, and the option to refresh the code can be
something<br>
selected from a menu on the slimp3 display.<br>
<br>
It'll mean some restructuring of the current server to support this,
but I<br>
wrote an IRC bot with Net::IRC this way, storing each possible IRC
event's<br>
code in a different file and when I added new features I just told the
bot<br>
to reload his own code and that way he never had to disconnect from
the<br>
server.<br>
<br>
<br>
<br>
Richard Smith<br>
richard-+***@public.gmane.org<br>
<br>
- Stay up to date with all your favourite news sources...<br>
- http://www.bearkey.com/alerts/<br>
- BBC, CNN, Slashdot, Wired, CNET and more.<br>
<br>
<br>
<br>
<br>
------------------------ Yahoo! Groups Sponsor
---------------------~--&gt;<br>
Sponsored by VeriSign - The Value of Trust<br>
Pinpoint the right security solution for your company - FREE<br>
Guide from industry leader VeriSign gives you all the facts.<br>
http://us.click.yahoo.com/pCuuSA/WdiDAA/yigFAA/rIp0lB/TM<br>
---------------------------------------------------------------------<span </span>~-&gt;<br>
<br>
To unsubscribe from this group, send an email to:<br>
slimp3-dev-unsubscribe-***@public.gmane.org<br>
<br>
&nbsp;<br>
<br>
Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ </blockquote>
<div><br></div>
</body>
</html>