... startfiles ....
 

soon after i made my posting about "startfiles" to the morpheus
forum , it got nuked and everthing was lost.

since I think that these startfiles are a simple but powerful way to
supply p2p sharing in favour of the user, I don't want it to be forgotten.
(The idea is not new at all, see http://www.museekster.com/index2.html)

If dont think, that this forum is the right place for it, please tell me.
Its a rather specific matter for fasttrack clients as Morpheus and Kazaa.

Startfiles are simply ".dat" files, as they are used by the clients to store
there search information.

Whats new now:
Since i don't like executable binaries, I wrote a perl script to create startfiles out
of the ".dbb" files. (1024k for now). It actually creates a .vbs script, which can be
reviewed and downloaded and which creates local .dat files.

Since the the script and dat files contain checksums and some file signatur they
offer some reliability that you get the correct file, which you do no have if you search
for it on the net. I will give some examples in a followup.

indy
:SH:

perl scripts
 

the attached zip contains the perl scripts used

perl gdat.pl perl-regex
searches the data1024.dbb for entry with perl-regex in filename
output is to stdout, you can redirect it to > xxx.vbs

perl csum.pl xxxx.dat
shows sum interesting parameters of a dat file

programming's just a bit quick and dirty.
i would appreciate if a perl programmer had a look at it.

indy.

janis joplin
 

janis joplin for beta testing

loreena mckennitt
 

loreena mckennit for beta testing

van morrison - no guru
 

van morrison - no guru .... for beta testing

Re: janis joplin
 

Kewl Dude!!:AP: :AP:

Gotta love Janis Joplin :love:

GR
*edited to let ya know I'll try the "Janet" as soon as I complete some toughies I've got going at the moment. I thought this an excellent idea when you first introduced it!

I'm also happy to see you've got some singles (the van morrison (whole cd) is too long for my slow conn).

Good going!

Re: ... startfiles ....
 

it's perfect here indy, it's what peer-to-peer is all about. thanks.:tu:

- js.

More searches
 

Well Done Indiana- Was following your theme on Morph, when it got nuked, great idea to re-post it on here, especially with the downloadable feature this board provides.

Here is another little trick that morpheus users may not know, to help you find who is contactable for downloads, browsing their files etc, Been around a while, but deliberately kept quiet, time to spill the beans after the way things have happened I reckon..

So, what to do? enter this phrase in the search box resume.dat

Start a search everything, you will see everybody that your system and super nodes can see, select a user, right click and do a get more from USER, this allows you to browse, wait for the list to finish before you try a column sort, Morpheus sometimes freeks out on sorting un-finshed lists.

Only problem is the single search screen, so you have to search all over again for the next list, but it does allow you to browse, and have a good look at whats there, even if you mate is connected.

Give it a try, but don't say I told you so, if it bombs your system?

Snark.

This is somewhat unrelated here, but I was getting some movies from Morph, I screwed up (again :D), and lost them.

I restarted the downloads on Morph, then went to iMesh to get the exact same things. The downloads on iMesh are coming 10 times faster than on Morph.

Once again, thanks to napho. :kiss:

Re: More searches
 

great tip snark, truly, :tu: and an easy one anybody can try. anymore like that back home?

- js.

I restarted the downloads on Morph, then went to iMesh to get the exact same things. The downloads on iMesh are coming 10 times faster than on Morph.


Sounds like Port capping at your ISP Dawn, Morpheus uses 1214 I think, don't know what port the iMESH client uses, anyone?

Snark.

Well Snark, I am quite blonde, but as far as I can tell, iMesh is also connected out on 1214.

Time to join the discussion about the startfiles that I have introduced at my website http://www.museekster.com.

I am impressed by the job that Indiana has done. I must confess that the way I make startfiles is less sophisticated.

This is how I do it.

Once I have discovered a high quality file, I again run a search with Grokster. After the identical file has shown up in my searchresults I cut off my internetconnection and hit the download button. Of course the download will not start but Grokster creates a datfile in your downloadfolder. That file is your startfile and ready for use.
The datfile is filled with a part of your free disk space and data like the usernames, filename and destination folder.

The free disk space that is included in this datfile may contain strange things.
Free disk space is not clean. It contains parts of files that have been deleted. When making a startfile this erased data becomes “active”. Part of video and music files are brought back to life. In a worst case scenario even a deleted virus can get activated and you spread it all around the planet in your startfile. Furthermore this data may prevent compressing the startfile to the smallest possible size.

I keep my free disk space clean with a utility called Eraser.
Eraser can be found at http://www.tolvanen.com/eraser/

One thing I have not figured out is how to edit these datfiles.
For instance changing the path or deleting usernames would be nice.
The datfiles can be opened with wordpad or a hexeditor but every change in the file is deadly. Maybe Indiana has a solution for this.


BTW. Nice board! The attachments option makes it a great place for exchanging startfiles.

the primary idea was, that the creator extrcts the startfiles,
posts them in the forum (best as text files) and everybody can copy them
and d/l the correct files from the start, or someone has verified files
and makes this afterwards.

why i preferred a forum is, that comments can be posted, if
something is good or bad or even a fake.

the .dat files contain a few things to "protect" the d/l mechnism.
(just like those old tombs - i'm used to :p)
what i think, there is a long (16/20) byte signature (MD5 like) which
is actually used together with filzesize to find a file - it corresponds
to a parameter which is in the .dbb file too.

the . dat file itself is protected by a 4 byte checksum
(the info how it works i got from a ex- morph forum poster)
if the checksum is somehow wrong, the .dat is deleted after
startup. therfore you cannot just edit the .dat.
the algorithm how to recalculate the checksum is contained in
the perl script in my second posting in this thread.

in principle, you just need filesize and signature to find a file
- and some script or program which makes a .dat out of it.

indy
:beer:

Hey, indiana_jones hopefully you've received my reply to your email. The startfiles are a good idea and I hope you continue with them. What users can also start doing is to give startfiles to each other through AIM, ICQ, etc.

I'm sure many users use AIM, for example, while they are on Morpheus. If startfiles become common knowledge then they could easily give them to their friends. And if you can't trust your friends to give you the correct files.......What we need to do is get the word out. Too bad the Morphues forums is currently down. I'm going to try and post some info about startfiles on my website and possibly link to other sites for more information. Do you know of any sites besides MuSeekster that explains startfiles?

You might have stated this, but how much of the file must be downloaded in order for it to work? All the dat files that I have recovered from a crash have been larger then 10 mb and they have resumed downloading without a problem.

Hi Johndoe.

Mind if I answer this? Startfiles are my invention basically. Indy took the creating of startfiles to a higher level.

Too bad I don't know a thing about perlscript so I have to stick to my own method.

They even work if you have downloaded nothing at all. Just check my post above. I contains a short how to.

I will soon add a manual in english on how to make startfiles to my website www.museekster.com.

As soon as there are more websites that offer startfiles I'll post them.

I'm just working on modifying my script to become a "startfile.exe"
unfortunatly the zipped version will be larger than 400k an cannot be
posted here, so if anybody wants to try it, please pm me you email,
if it's ready i will send it to you.

it will be and command line application, to be started in the
command (or MSDOS) window and will say something like that:

C:\Morpheus>startfile
startfile 1.0.a 2002 by Indy
based on the startfile idea from kewbase at http://www.museekster.com/
enter "startfile" without any parameters to get some help!

Usage: startfile "pattern"
Searches the files .\Db\data1024.dbb and .\Db\data256.dbb
for entries with filename that contain "pattern"
and writes "kazaadownloadxyz.dat" files to folder ".\Startfiles"

The downloads will be to "C:\StartfileDownload", but you can change this
settings by editing the ".\startfile.ini"

"startfile" should be run from the Morpheus folder

Since the .dbb files are locked, when morpheus is running,
you must either close Morpheus or
copy the *.dbb files to *.dbb.1 files, which are searched alternatively.

"pattern" is a perl regular expression:
case sensitive
. means any character (\. means .)
.* zero or more, .+ one or more of any character
example: "Janis Joplin.+Baby.*\.mp3"
see i.e. http://www.dcs.qmul.ac.uk/publicatio...erlRegexp.html


This exe file was created with the evaluation version of Perl2Exe.
For more information visit http://www.indigostar.com
(The full version does not display this message with a 2 second delay.)
...

indy

... the future starts exactly ........ NOW!!!:beer:

hi you all

dbb2dat.exe version 1.0.c

is available and produces startfiles, which at least are not at once deleted
by Morpheus.

additional alpha/beta/gamma tester are welcome.
cannot post the file here, because its to large, please pm your email address.

as an example I add the startfiles for an album from enya here.
the zip contains also .txt files, which hold the signature and file size.

because that's my next step:

sig2dat.exe

which can make a startfile from signature and filesize
when you see any on internet or get mailed you just type it in and get a .dat file.
such finding the correct files is done with a high reliabiltiy, but without searching.

filesize can easy be faked but the signature is 20 byte and calculated with a complex
algorithm - that means, if signature AND filesize are equal, it's as sure the same
as with fingerprint or genetic code.

i think, this would be interesting especially for movie and SW download.
maybe the signature could be also posted on that precious correct movie webbase

comments are welcome

indy
:beer:

Hey indy, good stuff you got going here.

Where does the signature come in? Where is it in the DAT file? Do you know how to calculate it?

hi AYB, glad to see you.

I must admit the "signature" is a little bit of a guess.
I read about fasttrack using some MD5 (additionally to your info) to
identify files.

see the "red" parameter type 3 below, same thing
in every dat. same string also in the .dbb file after download.

since MD5 sig is 16 Byte, I guessed it's something like that.
other parms, one can identify as some strings.
there are other binary values, which i do not know, what they mean.
some of them are some flags, for sure.

perl MD5 calculates a different result, but that doesn't say much. its
the only string in the .dbb, if there is nothing else.

indy


# ============================================================ ===
File: .\Startfiles\kazaadownload-Enya - Shepherd Moons - 04 - Ebuda.mp3.dat
Size of dat: 102922
Pointer: 420
CheckSum calculated: 17912884
CheckSum from File: 17912884
00000000
Sources: 1
# -- ---------------------------------------------------------------
Filename: Enya - Shepherd Moons - 04 - Ebuda.mp3 00 00
0e 3a 00 00 c3 0a 13 64 a8 24 53 55 2e 45 47 3d <.:.....d.$SU.EG=>
65 bd 02 77 4d af f6 83 <e..wM...>
Filesize: 2455731 [2398.174805] 002578b3
IP:Port, IP:Port: 206.142.53.203:1214, 137.73.4.105:1214
User: a1234@MusicCity.com 00
00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 <................>
00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 <................>
00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 <................>
00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 <................>
00 00 00 01 00 00 00 00 00 00 01 <...........>
# -- ---------------------------------------------------------------
Parameters: 6
Parameter Type 4 Length 6:
45 62 75 64 61 00 <Ebuda.>
Parameter Type 6 Length 5:
45 6e 79 61 00 <Enya.>
Parameter Type 8 Length 15:
53 68 65 70 68 65 72 64 20 4d 6f 6f 6e 73 00 <Shepherd Moons.>
Parameter Type 14 Length 8:
4e 65 77 20 41 67 65 00 <New Age.>
Parameter Type 26 Length 26:
45 41 43 20 2d 20 20 4c 61 6d 65 20 2d 2d 72 33 <EAC - Lame --r3>
6d 69 78 20 2d 62 31 31 32 00 <mix -b112.>
Parameter Type 3 Length 20:
c3 0a 13 64 a8 24 53 55 2e 45 47 3d 65 bd 02 77 <...d.$SU.EG=e..w>
4d af f6 83 <M...>
00 00 00 00 ff ff ff ff 00 00 00 00 <............>
Path: C:\StartfileDownload\Enya - Shepherd Moons - 04 - Ebuda.mp3 00
00 00 00 00 01 00 00 00 00 00 00 00 ff ff ff ff <................>
b3 78 25 00 00 00 00 00 00 00 00 00 00 4b 41 5a <.x%..........KAZ>
41 a4 01 84 28 91 17 <A...(..>
# == ========================================================

Shouldnt be too hard to work out how it is calculated with a little reverse engineering :BL:

I noticed your post says something about how many sources there are. How did you calculate this? I found it difficult to do so bcoz the user/filename strings seemed to be variable length?

it simply works, when you assume, that the data block after the filename is
of a fixed length (it should be around 141 Bytes(maybe +-1).
thus if you walk through it, adding

length(filename)+141

for each user (count is the 1. dw as you guessed), you get to the main data
block, which more or less resembles the data in the .dbb file

indy

hmm looks like i've overlooked some stuff in my examination of the DAT file. the repairer actually counts the number of sources by counting the number of @ surrounded by alphanumeric chars (the character after it can only be an M for musiccity, a K for Kazaa, a G for Grokster or ? for unknown)

so, if the pointer was corrupt and wouldnt go back far enuff, so u had to start at the end of the list (well, the end of the file, is it possible to find the end of the list from the end of the file?) how would you work your way back?

lets think of a kind of "matched" decoder

we have the rough structure for 1 found user:

byte 1 or 0 //always beetween users, has to be replaced by dw 1 later
string, byte 0 //filename + 0
bytearray [140] //maybe its 139 or 142
--- start of main block
dw nparms //doubleword number of parameters
loop nparms // repeated nparms
dw ptype1 // doubleword parametertype
dw plength // doubleword parameterlength
bytearray [plength] //parameter
end loop
bytearray [12] //unknown parms 12+-1
string downloadpath + byte 0 //filename downloadpath +0
bytearray [29] //unknown parms
string "KAZAAxxyyyy" //Kazaastring
EOF //end of file

just write a decoder or statemachine that resembles this structure,
then let it loop with an increasing offset from the
end of file, if it gets ready exactly with EOF - :beer:bingo:beer:.

at least you can try
indy
:J:

thx 4 possting in whats .DAT indy now ive take the the time to read this rapidly evolving thread
good work to all involved
will be getting pearl in the box in the next few weeks and trying to work this out
good work
kewbase
on
startfiles
do u reckon can i link to your page from morphia ?

Been poking around some DAT files and found some more info ( can't believe I missed it first time around :eek: ) do let me know if this is news to you indy ( also I've only got two DAT files to play with atm so I'm hoping this is applicable to all DATs )

I've been taking a look at the filename/user object and I've dissected it this far:

1. Filename. This is a variable length string of the filename as it is stored on the source's computer

2. 30 byte array. This appears to be another kind of signature or some such. Originally I mistook it for uninitialised memory/hard disk space but I noticed it is identical for every user

3. 16 byte array. Still not sure what this is. I can say unlike the previous array it does differ from user to user

4. 95 byte user name array. The username, unlike the filename appears to be fixed at 95 bytes.

And of course 95 + 16 + 30 = 141 a value which indy has already quoted to me.

That's all for now, I'm checking out the end of the file values which we haven't been able to identify yet

:cr:

PS: If ya need extra web space to provide a link for files that are too large
for the attachment feature on this forum, or other things if they're not too
big ...no problem ...let me know and I will upload them and tell you the link.

Also, I just got to thinking. Couldn't one make a self extracting zip file that
unzips to the download folder and automatically opens Morpheus when it is
finished? Like so>

u2boy.exe <click ...(the start files for one of my favorite albums of all time)...U2 - Boy

However...I suppose the download folder and program location would have
to be the default. Could it maybe "find" it? I'll leave that to you wizards!;)

PS: I made the start files like Kewbase said, but my zip file turned out to be kinda large.:dunno:

Finding the executable path and download directories is easy via the registry :D

Next version of my program MorpheusX ( http://go.to/MorpheusX ) acts as a loader and does this.

@AYB:
as far as i've analyzed the user field, it consists of the following

string filename, byte 0, byte 0 //filename +0+0
dw maybe_some_flags //dont knows, differs from user 2 user
bytearray[20] //signature same a parm 3 in main block
dw filesize //filesize in bytes
dw user-ip //
dw user-port //usually 1214
dw supernode-ip
dw supernode-port //usually 1214
string username, byte 0 //username
bytearray [141-tillthere] //don't know, differs

@Maze
the part of .dat, which contains the folder, where the downloaded
file is finally placed is protected through checksum, therefore, if
you will change this folder, you have to change it in dat and
recalculate the checksum.

indy

:f:

once again indy this is great work

also serves to highlight how incompetent :BB: these programmers are once again - port only needs to be a word

username definately seems to me to be fixed at 95 bytes ( you say it differs? :con: )

also, you say the dword following the double null terminating the string is different, on the DAT files I have they seem to be the same :con: but distinct from the 20 byte array bcoz this can also be found in the footer after the strings which are clearly song name, artist, description etc

:RS:

is morpheusX going to have some startfile idea involved ?
i sure you will let us know when it is released
but iam happy to try it out before that if you need another tester

It's possible MorpheusX will incorporate something involving start files ;)

Right now I barely managed to get v0.2 out by the end of january, which was my self imposed deadline :eek: and I've got plenty of work to do for uni so... need a couple weeks for things to clear up :tu:

program that views & manages .dat files
 

hi!

THANKX to all u for guessing all this info about the .dat files format... it has simplified my work... :TY:

i've developed a program that shows .dat info and let a user to obtain the downloaded data on various ways...
u all can try it at: http://www.angelfire.com/ego2/idleloop

(and i've complete a little the structure of .dat files... :ND: )

GreEtiNgs! :W:

GreEtiNgs idleloop!:tu: thanks for stopping in. nice site btw. the guys who invented ft sigdats are napsterites.net members and they've been known to help out w/info when they stop by from time to time. :beer:

- js.

:tu::beer::tu:
indy