Stratagus

New extraction program?

Asked by Bradley Clemetson

I have been working through Stargus and have been finding that there are files that are not being extracted (such as video) due to certain features not being available. While the same function I need is in the Wargus extraction tool. Which lead me to thinking that maybe we should think able about making a all-in-one tool game media extraction tool for multiple games. We could use Lua scripts (or specialized config) where developers can setup the specifics for their specific filesystem information. (Filepaths and Types) I was also thinking we should pack the files into a compressed archive (as a holder) with a ".stratagus" extension or the like so individual files have less a chance of getting mucked up or otherwise.

+Pros
++One tool for all games
++Functions that are commonly used will follow the same revision (MPQs, Smacker->Theora Conversion, pcx->png)
++Easier porting to different platforms

-Cons
--Maybe more difficult to tailor a specific game if implemented wrong (Map/Campaign Conversion)

What do you think? Corrections + Critiques?

Question information

Language:
English Edit question
Status:
Solved
For:
Stratagus Edit question
Assignee:
Bradley Clemetson Edit question
Solved by:
Bradley Clemetson
Solved:
Last query:
Last reply:
Revision history for this message
Kyran Jackson (erstmap) said : 		#1

Sounds like a reasonable idea. Good luck.

Revision history for this message
Joris Dauphin (joris-dauphin) said : 		#2

I agree that we could merge war1gus, wargus and stargus to share extractor tool, and some common scripts.
blizzgus ?

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#3

Should we just add another repository here on Launchpad?

Revision history for this message
Pali (pali) said : 		#4

I think that wargus extraction tool is working and bug free (I fixed all possible memory leaks & memory corruptions). Other active project is stargus and (maybe wargus). So do you really need some multi extraction tool for 2-3 games which already have (semi)working extractor? I think that there are other parts in project which can be improved... and I do not see reason for rewriting someting which is working.

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#5

I'm thinking that it would allow us to add new games to the engine easier (AOE,Command & Conquer, Command & Conquer: Red Alert), working with different editions (Warcraft II BattleNet Edition, Starcraft Anthology from the Blizzard Store) and handling different languages easier without having to write more tools that use many of the same functions. Convert format X to format Y, filesystem read/write routines. This could give more flexibility to other games to be extracted. (potentially to be as simple as writing a new script)

+It would also allow for performance increases such as multi-threading as we rid of nearly all if not all global vars and designing structures that allow for concurrency.
+We could design with the use of outside libraries (eg.Boost) to do the low level dirty work (filesystem paths, permissions checks, POSIX vs. WIN multithreading, exception handling, ect) so we don't have to update it ourselves when a OS deprecates that function call.

If we were to make this tool, it certainly take cues (functions) from the WarTool/StarTool for direction and we still want to maintain the current projects during transition. The big downside is the fact we are rewriting code but the big thing is whether we feel it would be better to take the short term but higher pain of doing the rewrite or take the long term but lower pain to track down the same/similar bug in throughout three (or however game extraction tools are created) codebases. We definitely would need to at first spend 90% of the time designing 10% coding and shift as the tool were to move forward.

Sorry for the wordy post

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#6

Plus we could get some really pretty cool UnitTesting.

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#7

I created a basic design doc (UML Diagram via Dia) on my site [https://codeprogrammers.net/index.php/projects/stratagus/29-strataextract-design-doc-alpha].

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#8

I forgot to mention I would really like feedback on it if possible. Including technical information of Warcraft II (DOS Version) and Warcraft data file types and formats. I have been reading through multiple articles on MPQ archives [http://www.zezula.net/en/mpq/main.html] ,methods of extraction and conversions from wav/smacker/ect using FFMPEG/Boost/Stormlib.

Revision history for this message
Bradley Clemetson (brad-c6) said : 		#9

I have created a Github repo for the moment for anyone interested in contributing.
https://github.com/bradc6/StrataExtract