Author Topic: Danmakufu Documentation Tool v1.8 (Read 9442 times)

SparklingOrange · « **on:** January 10, 2012, 03:47:53 AM »

I'm in the middle of creating a console application for Danmakufu: a function "dictionary" that allows you to search by keyword, index, or category, or simply inputting the full function name, which will then show you the full details of it. It's essentially like the "man" command in Unix/Linux, for those savvy with command line stuff.

The program is written in Java. I started last year (I was busy with school, so I couldn't get to it until now), when 0.12m was still being used, so when I found out that ph3 came out, I figured I'd put support on that as well. My only problem is that I'm grabbing text data off the wiki by hand (I meant for this to be an offline alternative in case Danmakufu Wiki goes down), and it's a really tedious process. If anyone can tell me a more efficient way of perhaps parsing data off the wiki to put into a file, that would be amazing.

Tool v1.2 (Java Command-Line tool for Windows/Mac/Linux)

Tool v1.8 (.NET GUI application for Windows only)

Chronojet ⚙ Dragon · « **Reply #1 on:** January 10, 2012, 04:03:48 PM »

So wait, you're going to pretty much create a version of Danmakufu Wiki? With explanations for functions and stuff for both v0.12m and ph3?

SparklingOrange · « **Reply #2 on:** January 10, 2012, 04:47:20 PM »

An offline version, yes. It should be fairly simple to use.

MMX · « **Reply #3 on:** January 10, 2012, 06:29:06 PM »

As if it could be attached to some text editor (like NP++) for context help, or danmakufu had it's own IDE :wikipedia:

SparklingOrange · « **Reply #4 on:** January 10, 2012, 06:50:15 PM »

Unfortunately, I could only create a standalone app that could be run on cmd (provided you set up a PATH for it for quick reference). I'd like to add support for NP++, but I still lack experience on that subject.

I'll add it in the future though, when I have more time. For now, I'll just release a vanilla version and update it when I can.

Blargel · « **Reply #5 on:** January 10, 2012, 09:57:00 PM »

If you want to make something for NP++, you'd have to generate some sort of ginormous XML file. And everyone knows XML is fun.

SparklingOrange · « **Reply #6 on:** January 11, 2012, 03:19:47 AM »

Sounds scary, but it's doable if I automated it

Chronojet ⚙ Dragon · « **Reply #7 on:** January 11, 2012, 05:35:44 AM »

Since some of the wiki is unfinished (translations to be done, garbage articles to be ignored, etc.) are you going to do those things as well?

KFCbbQ · « **Reply #8 on:** January 11, 2012, 07:59:05 AM »

Is the text data hard coded inside the program, or or you parsing it from separate files?

SparklingOrange · « **Reply #9 on:** January 12, 2012, 07:03:03 AM »

Quote from: alt-kyuu-kyuu on January 11, 2012, 05:35:44 AM

Since some of the wiki is unfinished (translations to be done, garbage articles to be ignored, etc.) are you going to do those things as well?

I'm gonna add them anyway, but when they do get updated, I'll update the files and re-release them ASAP.

Quote from: KFCbbQ on January 11, 2012, 07:59:05 AM

Is the text data hard coded inside the program, or or you parsing it from separate files?

I'm parsing them from separate files, which I copy-pasted from the wiki by hand. The files are purely text files with delimiters set up for proper formatting within the program.

I could hard-code them, but it'd be insane when there's approximately 355 functions that need to be processed for v0.12m and probably about 300 or so for ph3.

KFCbbQ · « **Reply #10 on:** January 12, 2012, 01:24:40 PM »

I decided to do some experiment on this because I haven't touched Java for a long time. So after toying for awhile I've created a semi-working parser which is able to convert the wiki pages into text data, by making use of the HTML syntax. Here are some examples of the text data it generates:

Bullet Control Functions
Motion Functions
Sound Functions

The source code is here.

The parser reads in from a text file (which is a raw HTML from a wiki page) and whenever a <h3> tag is encountered, the content wrapped inside the <h3 and </h3> will be formatted and is then treated as a function name. It will then extract the function's arguments and description from the following lines, and finally formatting them into the output file. The parser continues to scan until no more functions are found.

The trick here is to locate information using their corresponding HTML tags - function names are always wrapped inside <h3> and </h3>; arguments are wrapped between <pre> and </pre> ; descriptions are wrapped inside <p> and </p>. Note that sometimes the heading tags have parameters inside them, so instead of scanning for "<h3>", scan for "<h3" instead.

If you find this useful, perhaps you can use mine as a prototype and develop your own fully functional parser, although the process can be equally tedious than just by manually extracting them (or even end up wasting more time). But since your intention is to improve on your coding ability, i think it's a good exercise nonetheless.

SparklingOrange · « **Reply #11 on:** January 12, 2012, 03:27:43 PM »

I was considering making an HTML parser, but I figured it'd be faster (and I'd learn more about the functions myself) if I did it by hand. It's already been a third day, and it is taking rather long now (I'd be able to finish this in a day if I'm not in school full-time now). I'll play with your code and see what happens. I might use it in future updates for ph3 and put you as part of the credit

SparklingOrange · « **Reply #12 on:** January 13, 2012, 06:09:39 AM »

It appears I've hit a snag. After processing all that data for v0.12m, my files containing arguments, descriptions, and names respectively contain 363 lines each. With a smaller size, the files were processed properly without incident, but now, for some reason, my arguments file keeps derailing on the 55th line, and the Scanner function I'm using for it won't parse any more lines. The rest of the files seem fine, however... It's really weird. Does the Scanner function in Java have some sort of limitation I'm not aware of?

KFCbbQ · « **Reply #13 on:** January 13, 2012, 06:45:20 AM »

This sounds strange... Is it throwing any exceptions? I don't think the scanner will have problem handling just a few hundred lines of input though.

SparklingOrange · « **Reply #14 on:** January 13, 2012, 07:18:31 AM »

It was throwing a NoSuchElementException, which means it's run out of things to scan, even though there's still a lot. I was using the nextLine() method to grab the strings line by line, bur for some reason, one of the scanned lines cut off at a certain point, and the scanner failed to grab any more from there.

It could be because I was simultaneously scanning all three files at once (each had their own Scanner, of course). I might have to rework that part of the code, since I did some testing, and there was nothing wrong with the files themselves when I individually printlined each one. Though, I fear using too many lines for something as simple as scanning from a file and placing into memory.

KFCbbQ · « **Reply #15 on:** January 13, 2012, 09:54:01 AM »

Yeah, scan the files one by one if you can, that should fix the problem.

SparklingOrange · « **Reply #16 on:** January 13, 2012, 09:24:50 PM »

It just might be a hidden rogue character that's causing Scanner to fail...I'm not sure. It probably got carried over while I was copy-pasting because the arguments section of each entry was in a special boxed format.

I managed to work around this by replacing the Scanners into BufferedReaders. While I was redoing the code, I also ended up hard-coding the number of entries for each category, while omitting them from the files so they're just raw text data. It might take me a while until I can fix this and just a bit more to build it into a working .jar file for proper release.

By the way, I'm considering switching from NetBeans to Eclipse. Does Eclipse allow me to build in ant? For some reason, building in NetBeans only gives me the class files, but not an executable.

KFCbbQ · « **Reply #17 on:** January 14, 2012, 12:29:30 AM »

Class files are basically the executable format of Java, you can create a .bat with your text editor and type in
java class name
This will run the application when the .bat file is executed.

Also Eclipse is great you should try it out yourself, it can also generate .jar if that's what you want.

SparklingOrange · « **Reply #18 on:** January 14, 2012, 11:00:36 PM »

The tool is out now. Check the modified first post for download links.

SparklingOrange · « **Reply #19 on:** May 27, 2012, 07:35:08 PM »

Apologies for the bump, but I've updated the application some more. Download links are updated and such.

Look forward for a script for Notepad++ in the near future.

SparklingOrange · « **Reply #20 on:** March 10, 2013, 01:51:59 AM »

Final update for this application. Check the readme for details.

This application is now under the GNU GPLv3 license. I've attached my source code along the most recent release (v1.

and I am discontinuing maintenance on this app.

Good day to you all.

Sparen · « **Reply #21 on:** March 10, 2013, 03:35:29 PM »

Quote from: Blargel on January 10, 2012, 09:57:00 PM

If you want to make something for NP++, you'd have to generate some sort of ginormous XML file. And everyone knows XML is fun.

The issue with that is that it can only be run easily on Windows. Notepad ++ works in wine, but... you can't set it as the default application to open .dnh files or .txt files because it's not a native application.