Author Topic: Testing out solutions to Touhou 13.5 patch?  (Read 391428 times)

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #30 on: June 03, 2013, 10:04:39 AM »
I've been looking at this thread and I was just wondering, is there work being done on a patch or is everything shown just conceptual?

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #31 on: June 03, 2013, 12:07:55 PM »
First off, sorry if this is a bit unrelated to the topic, I can't create topics and the HM data unpacking discussion was split from the main HM thread into here.
Are TFCSTool and TFBMTool available for download anywhere yet, or will they be made public in the near future? I really like HM's music and I would enjoy being able to listen to it on my computer (without having to use the musicroom), and on my phone. I've searched the internet for hours for a solution and haven't found any program capable of extracting data from the .pak/TFPK format, probably due to my inability to understand Japanese/Chinese.

Any help would be greatly appreciated.

Savory

  • I am a save frog
  • *wink*
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #32 on: June 13, 2013, 02:21:47 PM »
Any updates on this, yet?

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #33 on: June 13, 2013, 10:55:13 PM »
Any updates on this, yet?
That's what I'm wondering, too.
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Ciel

  • Green-Eyed Jealousy
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #34 on: June 14, 2013, 11:46:59 AM »
That's what I'm wondering, too.
I'm pretty much wondering about this too as well. =];; *patiently waiting it out~*

Are TFCSTool and TFBMTool available for download anywhere yet, or will they be made public in the near future?
This... is the first time I ever heard about these things. O_O; What do they do?

Just some weirdo, move along!

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #35 on: June 14, 2013, 06:57:42 PM »
This... is the first time I ever heard about these things. O_O; What do they do?
Those are the tools shown in the screenshots in cuc's second post, they extract/repack 13.5's data files.

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #36 on: June 14, 2013, 09:32:41 PM »
Ooh, that's what they are. Now that you mention them, I do wonder if they're up yet, so we can start work.
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #37 on: June 16, 2013, 03:21:28 AM »

Ciel

  • Green-Eyed Jealousy
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #38 on: June 16, 2013, 09:49:57 AM »
credit to: http://www.shrinemaiden.org/forum/index.php/topic,197.msg985230.html#msg985230

go to this site

https://github.com/nyfair/foo_thbgm/commit/258531875b79f3dfa741be637f250091d6df777c

see
Code: [Select]
+// No interest in attaching tfpk unpacker until tasfro make enough $$$ haha
Eh? Is that one of the developers for the unpacker? I hope they are willing to share with us soon.  D:

Either way, It seems that the coding for HM is extremely complicated... Fingers crossed to the developers...;;

Just some weirdo, move along!

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #39 on: June 16, 2013, 07:36:10 PM »
Here is all toolkits we used in HM Chinese patch.
https://dl.dropboxusercontent.com/u/15824386/135tk.zip
Notes:
1. The game itself used RSA as .pak file list encryption algorithm, since I cannot get d, we have to patch the game exe itself if we want to use repacked packages. /h maybe useful.
2. Only hashed path&names remained in .pak file list, so there maybe some UNK_(through seems like all files needed in localization have correct path), sorry for inconvenience.
3. TFBMTool currently supports only 32-bit images since there are no text on 8-bit with palette images.
4. All scripts(including which used to draw ballon dialog) game used are Squirrel 3.0.4, unfortunately there are no working decompilers so it's hard to change it's behaviour. But cnuttool can efficiently extract&replace literals in cnut.
5. ACT files are quite messy and sorry for this.
6. Font rendering in HM made me a bit confused even for Chinese characters....

Tell me if anyone interested in codes. (it is VERY MESSY and looks UGLY :(

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #40 on: June 17, 2013, 02:49:07 AM »
Here is all toolkits we used in HM Chinese patch.
https://dl.dropboxusercontent.com/u/15824386/135tk.zip
Notes:
1. The game itself used RSA as .pak file list encryption algorithm, since I cannot get d, we have to patch the game exe itself if we want to use repacked packages. /h maybe useful.
2. Only hashed path&names remained in .pak file list, so there maybe some UNK_(through seems like all files needed in localization have correct path), sorry for inconvenience.
3. TFBMTool currently supports only 32-bit images since there are no text on 8-bit with palette images.
4. All scripts(including which used to draw ballon dialog) game used are Squirrel 3.0.4, unfortunately there are no working decompilers so it's hard to change it's behaviour. But cnuttool can efficiently extract&replace literals in cnut.
5. ACT files are quite messy and sorry for this.
6. Font rendering in HM made me a bit confused even for Chinese characters....

Tell me if anyone interested in codes. (it is VERY MESSY and looks UGLY :(
TIME FOR ACTION.

No need to be sorry for anything, this looks like it will be so helpful. Thanks so much!

EDIT: Now could you explain how it works in general? I already have a general idea of how it works, andI just need a little push before I dig in.
« Last Edit: June 17, 2013, 02:54:28 AM by Lukesky180 »
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #41 on: June 17, 2013, 03:54:51 AM »
TIME FOR ACTION.

No need to be sorry for anything, this looks like it will be so helpful. Thanks so much!

EDIT: Now could you explain how it works in general? I already have a general idea of how it works, andI just need a little push before I dig in.

Hmm what does "how it works" mean?
For usage just run them under terminal or see batch files included in zip.

(If you have no idea about what I said below, just ignore)
And for how to figure out the game files format... well just repeatly observe the file itself, debug or IDA the game and do some guessing until you found enough info. then code a tool.
I haven't written formal documentations about the file format.All notes made during the progress is in Chinese ><
See my messy codes if you are really interested in internal file formats.
WARNING: Most of codes are written as "one-time" solution and maybe wrong or misleading(especially for type "wtf" in .ACT, actually it has different meaning in different layouts)

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #42 on: June 17, 2013, 11:55:24 AM »
Hello Riatre,

thank you very much for your generosity by sharing the tools and source code! This is certainly a very helpful contribution for the community's translators, for those who like messing around with games, and for those who enjoy the game's music.

For anyone having problems, here are some rough instructions on how to use these tools to translate the game:

The first thing to do is run "th135arc.exe /x th135.pak". This will extract all files from the th135.pak file into a subfolder called "th135".
Some of the extracted files will still be compressed with zlib, the TFBM format for bitmap images, and the TFCS format for .csv files (containing in-game text).
Run "extractBM.bat" and "extractCS.bat" to automatically decompress all supported files in the subdirectories into an easily editable PNG format. This process may take a while for the image files, and it will not extract the 8-bit-paletted character sprites (which are irrelevant for translation anyway). Also, some files may be skipped due to "invalid filename", this is because of locale problems with the file names and should be fixed by AppLocale (can't test this since I am on Wine at the moment).

Translating the image files should be fairly trivial, you'll find most text images in the "System" folder.
The .csv files are text files in Shift-JIS and not only allow you to translate names and descriptions, but often also control game functionality, allowing you to change skill/spell card script functions, mana cost and faith values, as described by the first line of each file. I'm tempted to just mess around and mix characters' cards up a bit, but I don't want to risk corrupting my th135.pak at the moment, and this is pretty much unrelated to translating the game anyway.
The .nut and .act files can be decompressed using "dumpcnut.bat" and "dumpact.bat", respectively, and seem to contain strings related to Squirrel scripts. These should probably be left alone for now.

Before continuing, make a backup of your th135.pak and th135.exe. This step may corrupt the file in some way if you haven't set up your locales properly.
To repack the archive, run "repackBM.bat" and "repackCS.bat" (and optionally "writecnut.bat" and "writeact.bat"), then run "th135arc.exe /p th135", followed by "th135arc.exe /h th135.exe" to change the security hashes in th135.exe and allow the changed th135.pak to work. This worked partially for me: the game still runs and displays most images, but stage backgrounds do not work anymore. To revert to the original version of the game, restore the backups of both th135.pak and th135.exe. Restoring only the th135.pak file and then running "th135arc.exe /h th135.exe" again does not seem to work at the moment.
« Last Edit: June 17, 2013, 11:57:00 AM by Marukyu »

Ciel

  • Green-Eyed Jealousy
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #43 on: June 17, 2013, 12:07:18 PM »
Omg it came out! Thank you very, VERY much!

... But uhh... I... kind of have a problem at Step 1

http://i.imgur.com/hyF8FzX.png

Guess it's a dll problem. ^^; I'm trying to find a good one but if anyone has it, mind PMing me it? TY!

EDIT: While I'm sure there's more than 3 of us being eager to work on this now... I propose we make a small role call team to see who does what, before some of us end up doing the same thing, wasting one of our time. D:
« Last Edit: June 17, 2013, 12:09:22 PM by Ciel »

Just some weirdo, move along!

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #44 on: June 17, 2013, 12:25:22 PM »
Don't forget.
there is DDS files too.

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #45 on: June 17, 2013, 12:39:20 PM »
Omg it came out! Thank you very, VERY much!

... But uhh... I... kind of have a problem at Step 1

http://i.imgur.com/hyF8FzX.png

Guess it's a dll problem. ^^; I'm trying to find a good one but if anyone has it, mind PMing me it? TY!

EDIT: While I'm sure there's more than 3 of us being eager to work on this now... I propose we make a small role call team to see who does what, before some of us end up doing the same thing, wasting one of our time. D:

Not entirely sure about the DLL problem, but you could try downloading the MS Visual C++ Runtime 2012 (or its 64-bit version); a functional version of that DLL should be included in there. My version of it is a dummy DLL by Wine, which unfortunately wouldn't work if I sent it.

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #46 on: June 17, 2013, 12:42:41 PM »
Omg it came out! Thank you very, VERY much!

... But uhh... I... kind of have a problem at Step 1

http://i.imgur.com/hyF8FzX.png
Guess it's a dll problem. ^^; I'm trying to find a good one but if anyone has it, mind PMing me it? TY!

EDIT: While I'm sure there's more than 3 of us being eager to work on this now... I propose we make a small role call team to see who does what, before some of us end up doing the same thing, wasting one of our time. D:

Maybe you need this
http://www.microsoft.com/en-us/download/details.aspx?id=30679


Hello Riatre,

thank you very much for your generosity by sharing the tools and source code! This is certainly a very helpful contribution for the community's translators, for those who like messing around with games, and for those who enjoy the game's music.

For anyone having problems, here are some rough instructions on how to use these tools to translate the game:

The first thing to do is run "th135arc.exe /x th135.pak". This will extract all files from the th135.pak file into a subfolder called "th135".
Some of the extracted files will still be compressed with zlib, the TFBM format for bitmap images, and the TFCS format for .csv files (containing in-game text).
Run "extractBM.bat" and "extractCS.bat" to automatically decompress all supported files in the subdirectories into an easily editable PNG format. This process may take a while for the image files, and it will not extract the 8-bit-paletted character sprites (which are irrelevant for translation anyway). Also, some files may be skipped due to "invalid filename", this is because of locale problems with the file names and should be fixed by AppLocale (can't test this since I am on Wine at the moment).

Translating the image files should be fairly trivial, you'll find most text images in the "System" folder.
The .csv files are text files in Shift-JIS and not only allow you to translate names and descriptions, but often also control game functionality, allowing you to change skill/spell card script functions, mana cost and faith values, as described by the first line of each file. I'm tempted to just mess around and mix characters' cards up a bit, but I don't want to risk corrupting my th135.pak at the moment, and this is pretty much unrelated to translating the game anyway.
The .nut and .act files can be decompressed using "dumpcnut.bat" and "dumpact.bat", respectively, and seem to contain strings related to Squirrel scripts. These should probably be left alone for now.

Before continuing, make a backup of your th135.pak and th135.exe. This step may corrupt the file in some way if you haven't set up your locales properly.
To repack the archive, run "repackBM.bat" and "repackCS.bat" (and optionally "writecnut.bat" and "writeact.bat"), then run "th135arc.exe /p th135", followed by "th135arc.exe /h th135.exe" to change the security hashes in th135.exe and allow the changed th135.pak to work. This worked partially for me: the game still runs and displays most images, but stage backgrounds do not work anymore. To revert to the original version of the game, restore the backups of both th135.pak and th135.exe. Restoring only the th135.pak file and then running "th135arc.exe /h th135.exe" again does not seem to work at the moment.
To avoid "invalid filenames", try to run th135arc with AppLocale. Most background images have Japanese character in it's name :(
It looks like this bug is caused by taking current code page (in OS language settings) for converting ANSI filenames to Unicode and got '??????' in your environment.
« Last Edit: June 17, 2013, 01:07:36 PM by rinkoshrine »

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #47 on: June 17, 2013, 02:09:43 PM »
Now to work on some English proof of work. I'm very excited about this, it has revived our translation efforts.
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Ciel

  • Green-Eyed Jealousy
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #48 on: June 17, 2013, 04:05:43 PM »
I'm not sure what happened to the Private E-Mail inbox, since there isn't anything appearing in my Sent Messages folder, but in case you two didn't get it...

Luke, Marukyu, do you want to share your Skype/AIM contacts with me? =o Or anyone else who wants to help out? 8D (Since there isn't a group yet... Hm.) I kind of want to talk more about HM which might flood the forum ahaha;;

Anyways, I took a whirl on the tools (Thanks Maru and Rinko!) and I managed to edit a few things. Like Maru, the stage backgrounds do not work for me, and the character portraits are blank.




So far, I did the character names and the main menu. Translation is most likely inaccurate. Nothing too big since I don't know how to open the message files...;; (Profile 1 and COM at the bottom are outlined white too. I don't think I touched them...)

I wanted to put their full character names, but I am unable to manipulate the length unless you fix it in the coding, I think.

And then I did the opening too. Works just fine! (Except the main menu, the background is black but I wanted to make it look nice-)



... WELL, overall, uhh... PROGRESS... RIGHT?

Edit: Took a while to upload, but I also attempted to translate the opening!

http://www.youtube.com/watch?v=to9AUxk5ts8

The menu at the end was supposed to be black (And I fixed that position at the second line...) but... aha, that's how the opening is, currently. Opinions please? >A</
« Last Edit: June 17, 2013, 04:36:56 PM by Ciel »

Just some weirdo, move along!

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #49 on: June 17, 2013, 04:51:07 PM »
To avoid "invalid filenames", try to run th135arc with AppLocale. Most background images have Japanese character in it's name :(
It looks like this bug is caused by taking current code page (in OS language settings) for converting ANSI filenames to Unicode and got '??????' in your environment.
This has to be some bug with Wine; I'm using Linux. Even though I set it up to use the Japanese locale, it corrupts the filenames. I can try to port/compile Linux versions of the tools (if you're okay with that, of course), that might work better in my case.

Like Maru, the stage backgrounds do not work for me, and the character portraits are blank.
It's a filename issue, according to Riatre. You could try if AppLocale works with the tools and see if it results in correct filenames.

Ciel

  • Green-Eyed Jealousy
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #50 on: June 17, 2013, 05:26:16 PM »
It's a filename issue, according to Riatre. You could try if AppLocale works with the tools and see if it results in correct filenames.
Yep! Applocale did it! I was a bit confused since I'm using Windows 7, and I thought the system locale took care of it. D:

Thanks for the help, you two! I'll try to do more tomorrow!

(... that is if I can do anything. Could we have a role call please? =])

Edit: Scratch that, Applocale didn't;; Back to the board. D:
« Last Edit: June 17, 2013, 06:24:50 PM by Ciel »

Just some weirdo, move along!

Quwanti

  • Pixel stuff
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #51 on: June 17, 2013, 05:45:52 PM »
So you appear to be able to extract the images.

Would you be so kindly to release a full sprite rip? :D I myself can't, because the th135arc.exe (and actually any other exe which uses some kind of command prompt) automaticly closes in one blink.

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #52 on: June 17, 2013, 06:02:56 PM »
So you appear to be able to extract the images.

Would you be so kindly to release a full sprite rip? :D I myself can't, because the th135arc.exe (and actually any other exe which uses some kind of command prompt) automaticly closes in one blink.

TFBMTool does not support the 8-bit sprites and their palettes yet. I could try to extend it to support them, if Riatre is okay with that. About the programs closing instantly: start the command line (run cmd.exe), enter "cd C:\Path\To\Th135\" and then enter the commands specified in my post.

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #53 on: June 17, 2013, 06:15:35 PM »
So you edit the text graphically?
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #54 on: June 17, 2013, 06:28:19 PM »
So you edit the text graphically?

It depends. Some text data in the game is stored in csv files (items/cards, story mode dialogue, music room descriptions), which are comma-separated plaintext files, whereas some other text is stored as images (menus, character names).

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #55 on: June 17, 2013, 07:00:12 PM »
Alright, after extracting the files, I need help. I can't edit the PNG files, no matter what I do. It says the file is "unsupported"?
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #56 on: June 17, 2013, 07:03:33 PM »
Alright, after extracting the files, I need help. I can't edit the PNG files, no matter what I do. It says the file is "unsupported"?

That's what you need to run "extractBM.bat" for, it decompresses all supported image files. Same goes for csv files, except you use "extractCS.bat" for those.
When you're done, use "repackBM.bat" and "repackCS.bat" to compress the files back into the original format.

Extracting and repacking the image files might take a few minutes.

Lukesky180

  • Knight of the Hungry Ghost
  • Yuyuko-sama!
    • HSE-Endeavor's Robotics Team
Re: Testing out solutions to Touhou 13.5 patch?
« Reply #57 on: June 17, 2013, 07:06:12 PM »
That's what you need to run "extractBM.bat" for, it decompresses all supported image files. Same goes for csv files, except you use "extractCS.bat" for those.
When you're done, use "repackBM.bat" and "repackCS.bat" to compress the files back into the original format.

Extracting and repacking the image files might take a few minutes.
Ah, stupid me. Thanks so much!

I guess I was in such a rush that I forgot to do that step, just the initial one. Thanks again!

EDIT: Anyways, as I'm waiting for everything to decompress, what fonts do the images use?
« Last Edit: June 17, 2013, 07:31:10 PM by Lukesky180 »
>This is the little mistress.
>Oh, hello little one!
Is this my dream, the destiny to behold?
~Sister of Scarlet~

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #58 on: June 17, 2013, 07:31:17 PM »
Ah, stupid me. Thanks so much!

I guess I was in such a rush that I forgot to do that step, just the initial one. Thanks again!

EDIT: Anyways, as I'm waiting for everything to decompress, what fonts do the images use?

No problem! Glad I could help.

It probably doesn't really matter what font will be used in the translated images (as long as it's readable and looks good), because even if the translation used the same fonts as the original, the Latin characters would still look different from the Japanese characters, so there probably isn't much of a point in putting time and effort into finding the exact font used for the game. In my opinion, it's much more important to stay consistent within the translation in terms of fonts and text effects (outlines, shadows, etc).

Re: Testing out solutions to Touhou 13.5 patch?
« Reply #59 on: June 17, 2013, 07:34:15 PM »
I could try to extend it to support them, if Riatre is okay with that.

Thank you and feel free to do all you want  :)