Maidens of the Kaleidoscope

~Hakurei Shrine~ => Rika and Nitori's Garage Experiments => Topic started by: Areylie on January 23, 2010, 03:49:36 AM

Title: Danmakufu .dat File Packer/Unpacker [v2.0]
Post by: Areylie on January 23, 2010, 03:49:36 AM
===th_dnh_archiver v2.0===

Execute <dnharc> or <dnharc.bat> for the command line interface.
Execute <th_dnh_archiver> or <th_dnh_archiver.bat> for the graphical interface.

[attach=1]
[attach=2]
Source code can be found in <src.zip>. Licenced under the GNU General Public Licence v3 or later

Changelog v0.5.0:
  * First release.
Changelog v0.5.1:
  * Fixed typo.
Changelog v0.5.2:
  * Changed code to allow non-continuous file data. (e.g. file data being stored as [file 1][file 3][file 2] instead of [file 1][file 2][file 3])
  * Made the code much more efficent.
  * Implemented the -o flag.
Changelog v1.0:
  * Implemented compression.
  * Fixed fatal bug causing the corruption of almost all extracted files introduced in v0.5.2.
Changelog v1.0.1:
  * Fixed a bug where you couldn't compress directories with subdirectories in them.
Changelog v2.0:
  * Rewritten.
  * Added graphical user interface.

Known Issues:
  * Version v2.0 is currently slower than v1.0.1
  * Due to a hack in place to deal with unacceptably slow compression speeds, you cannot compress files larger than 2GB.



==Danmakufu .dat archive format==

[magic number, 5041434B5F46494C4500]
[4 bytes, little endian, number of files]
[12 bytes*number of files+total length of names (with null terminator), file definitions]
[can be any size, file data]

The format for file definitions is the following:
[4 bytes, little endian, length of name][name, null terminated]
[4 bytes, little endian, offset]
[4 bytes, little endian, file length]

Also, files that begin with [434F4D50524553535F5A495000] are compressed. The format for compressed files follows:
[magic number, 434F4D50524553535F5A495000]
[4 bytes, little endian, uncompressed size of data]
[can be any size, compressed data, compressed using the deflate algorithm]
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Drake on January 23, 2010, 03:50:56 AM
Oho, so you cracked it, or does it just randomly compile? Can you play full scripts with just the archive?
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 03:56:55 AM
Full scripts do run with just the .dat file as far as I can tell.
Also, http://www.geocities.co.jp/SiliconValley-Oakland/9951/products/th_dnh_archiver.zip (http://www.geocities.co.jp/SiliconValley-Oakland/9951/products/th_dnh_archiver.zip)
There's a offical packer, though I don't think there's any unpackers yet, other than the almost-done one I have. (Unless it's hidden in a corner of the Japanese wiki)
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 04:37:44 AM
v0.5 released.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 04:42:27 AM
v0.5.1 released.

Changelog v0.5.1:
  * Fixed typo.

... ::)
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:16:13 AM
(http://img230.imageshack.us/img230/5993/loljava.png)

hurrrrr
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:17:35 AM
A possible solution to the issue is to install Java.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:20:22 AM
It would be, if it weren't already installed.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:22:48 AM
You probally have your classpath set up wrong.
http://java.sun.com/docs/books/tutorial/essential/environment/paths.html (http://java.sun.com/docs/books/tutorial/essential/environment/paths.html)
http://java.sun.com/j2se/1.3/docs/tooldocs/win32/classpath.html (http://java.sun.com/j2se/1.3/docs/tooldocs/win32/classpath.html) woops wrong classpath
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:26:28 AM
Just reinstalled to be sure, and it still doesn't work.

I wouldn't know about class paths, I just had it installed initially so I could run web games and such, like everyone else. You'd figure that the installer would have taken care of that, and this is the first time a program that uses Java has messed up on me like this.  :V

Just going to throw this out there, but I'm running Windows XP 64-bit edition. That's thrown a wrench in a few other programs before, maybe it's relevant, I dunno.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:27:23 AM
Yeah, the Java installer dosn't set up paths. Might add instructions to the next version's readme. :/

Is the error the same? XP handles drag and drop wierd. If so, try copying th_dnh_archiver.jar to "C:/Documents and Settings/*Your username here*"

P.S. "classpath?" I mean "path." Classpath is a java term.  =P
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:29:23 AM
Yeah, I would have put in another image if the error had changed. Taking a look at that path stuff now...

Quote
To make a permanent change to the CLASSPATH variable, use the System icon in the Control Panel. The precise procedure varies depending on the version of Windows.

The PATH can be a series of directories separated by semicolons (;). Microsoft Windows looks for programs in the PATH directories in order, from left to right. You should have only one bin directory for the JDK in the path at a time (those following the first are ignored), so if one is already present, you can update that particular entry.

Step 1: Open system.
Step 2: ???
Step 3: Profit!



wonderful

...which file am I supposed to point the thing to, anyways? Those instructions mention a "javac MyClass.java", in Java\jdk1.6.0\bin\, but I haven't found that file in any of the folders most resembling the example(jre6\bin\, jre1.6.0_03\bin\, jre1.6.0_05\bin\, jre1.6.0_07\bin\). Bleh.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:30:57 AM
You can also modify the .bat files to use the full path to java.exe, instead of just "java"

Wait, do you even know where Java is installed to? `-`
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:38:06 AM
Yeah, I would have put in another image if the error had changed. Taking a look at that path stuff now...

Step 1: Open system.
Step 2: ???
Step 3: Profit!



wonderful
http://www.computerhope.com/issues/ch000549.htm (http://www.computerhope.com/issues/ch000549.htm)
wonderful
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:42:58 AM
Wait, do you even know where Java is installed to? `-`

More like I have a Java folder with 4 directories, all with their own bin subdirectories and Java executables. I'm going to try the one in jre1.6.0_07.

EDIT: well, I added it as both a system and a user variable. Same error.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:46:29 AM
Replace "java" in the bat file with the full path to java.exe
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 05:53:31 AM
Unable to find th_dnh_archiver.jar. I even copied it to my documents folder like you said, no dice.

EDIT: and on a wild hunch, I copied it over into the directory where the Java executable was. Naturally, it didn't work.  :V
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 05:56:57 AM
Unable to find th_dnh_archiver.jar. I even copied it to my documents folder like you said, no dice.
Not the Documents folder. Your user directory. Press "Go Up One Directory", or whatever is equivalent.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 06:01:26 AM
Okay, Decompression complete.

So where did the decompressed files get put? :E
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 06:02:35 AM
All over your home directo- jk, The directory "output" in the same directory that th_dnh_archiver.jar is in.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 06:04:43 AM
Ah, I see...It worked on th_dnh.dat.


...now if only it worked on the Cconcealed the Conclusion dat files, then I'd really be in business.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Areylie on January 23, 2010, 06:06:52 AM
Those are common data files. Not .dat archives.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Prime 2.0 on January 23, 2010, 06:09:43 AM
Ah. They use the same extension, so I thought that those might have been more or less the same thing...


However, if you were to make something to let me properly view and alter said common data files, that would be really awesome(not that what you've done already isn't pretty sweet).
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5]
Post by: Stuffman on January 23, 2010, 08:31:33 AM
Merged the file format specifics thread to save a bit of space, since there's not much discussion to be had on that subject alone.

Also, this topic is now in the Tutorial Index misc. information section.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Areylie on January 24, 2010, 09:57:48 PM
v0.5.2 released.

Changelog v0.5.2:
  * Changed code to allow non-continuous file data. (e.g. file data being stored as [file 1][file 3][file 2] instead of [file 1][file 2][file 3])
  * Made the code MUCH more efficent.
  * Implmented -o flag.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Infy♫ on January 25, 2010, 07:13:08 PM
i wonder what the use of this is, if you can actually unpack everything.
the original archiver existed so people wouldn't steal the scripts you made
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Azure Lazuline on January 26, 2010, 12:33:46 AM
Yeah, this seems like it exists solely to steal other peoples' work. If they wanted you to access the files, they would have released it as a folder instead of an archive... I'm all for everything being open-source too, but I still think the author's preference should come first.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Areylie on January 26, 2010, 03:12:32 AM
If anyone else objects, I could take it down. Anyways, I doubt anyone could really steal other's work with this, one reason being that it's hard as hell to reconstruct the folders after decompression- Danmakufu ignores directories while dealing with .dat packed files.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Areylie on February 07, 2010, 01:47:56 AM
The bug has been fixed. I'm going to write code to create archives, and then I'll upload that.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Areylie on February 07, 2010, 06:32:13 AM
v1.0 released.

Changelog v1.0:
  * Implmented compression.
  * Fixed fatel bug causing the corruption of almost all decompressed files. (that was introduced in v0.5.2 - woops)
Title: Re: Danmakufu .dat File Packer/Unpacker [v1.0]
Post by: Areylie on February 07, 2010, 04:52:31 PM
v1.0.1 released.

Changelog v1.0.1
  * Fixed an asdfasdfsdf bug where you can't compress folders with subdirectories in them.
Title: Re: Danmakufu .dat File Packer/Unpacker [v0.5.2]
Post by: Chronojet ⚙ Dragon on June 05, 2010, 05:04:18 AM
Yeah, this seems like it exists solely to steal other peoples' work. If they wanted you to access the files, they would have released it as a folder instead of an archive... I'm all for everything being open-source too, but I still think the author's preference should come first.
Yes. I think Cirno-chan had this motivation to peek into peoples' scripts...
I feel less confident in posting my scripts now.

Also, I still don't get how someone would even use this program... -thing. I pretty much follow the instructions in the first post (was it the first post?). Then I looked in the replies for anyone with the same problem. Prime 2.0 already fixed HIS problem... but I got lost on the way. :ohdear:

Also, why does Azure's picture look like Satori? :derp:
Title: Re: Danmakufu .dat File Packer/Unpacker [v1.0.1]
Post by: NekoMikoSanae on July 02, 2010, 03:31:37 PM
Apologies for bump in advance.

When I drag the file to the decrompressers I get this:

"Unable to access jarfile th_dnh_archiver.jar
Press any key to continue . . ."

I have no experience at all with Java. If that's the problem, please explain it like you'd explain to a six year-old. D;
Title: Re: Danmakufu .dat File Packer/Unpacker [v1.0.1]
Post by: Areylie on March 25, 2011, 08:13:48 AM
v2.0 released.

Changelog v2.0:
  * Rewritten.
  * Added graphical user interface.
Title: Re: Danmakufu .dat File Packer/Unpacker [v2.0]
Post by: Zengar Zombolt on March 25, 2011, 01:32:56 PM
  * Added graphical user interface.
YESSSSSSS
I fucking love GUIs. Thanks! o/