Getting all icons by ID

04/05/2012 12:09 ItsMeMario1#1

Oke so i have been trying to figure this out, I want to get all the icons and save them as separate image by ID. This because elements.data links items with icon ids. i know the icons are saved in surfaces/iconset folder and getting the images is not the problem. reading the txt file isnt the problem either. my problem is how can i link the Chinese name (or whatever those strings are XD) in the txt file with a item id so i can connect the dots.

where would i have to look to find out out this 2 dots.

btw this is for ESO and i know nobody gives a crap about eso here but this should work the same as in PW

and wewt... finally my first post xD

04/05/2012 14:01 dumbfck#2

I have some tools somewhere in my archives that might be able to help. Not sure which hard drive they're on or where they are but I'll try to find them later this evening (got a lot to do today :P).
I'm not familiar with ESO though... Is it similar to PW whereby there is a large .dds file and a text file with a massive list of chinese stuff?

I've got a tool which extracts all the images and a separate tool which is sort of a C# implementation of sELedit that extracts elements.data to a local .sdf database. This database then contains all the item info such as ID, surface file name etc.
However, it relies on the config file used for sELedit being up to date... I haven't checked in a while so I'm not sure if the author of sELedit is still maintaining that.
I'll take a better look at everything later :P

Edit:
About 10 minutes after my previous post, I realised the obvious flaw in my suggestion about the elements.data extractor... That's only ever going to work for the PWI elements.data file, not ESO.

I know that in PWI, in an item struct there is value which gives the positional offset of the icon in the big .dds icon list (or is that only in inventory slots? I can't remember right now). It would possibly make sense that there is perhaps a massive list of all in-game items in memory, however, I don't know where that is in PW, let alone ESO. That might be a path worth investigating though.

04/05/2012 20:11 ItsMeMario1#3

Quote:

Originally Posted by dumbfck

I have some tools somewhere in my archives that might be able to help. Not sure which hard drive they're on or where they are but I'll try to find them later this evening (got a lot to do today :P).
I'm not familiar with ESO though... Is it similar to PW whereby there is a large .dds file and a text file with a massive list of chinese stuff?

I've got a tool which extracts all the images and a separate tool which is sort of a C# implementation of sELedit that extracts elements.data to a local .sdf database. This database then contains all the item info such as ID, surface file name etc.
However, it relies on the config file used for sELedit being up to date... I haven't checked in a while so I'm not sure if the author of sELedit is still maintaining that.
I'll take a better look at everything later :P

Edit:
About 10 minutes after my previous post, I realised the obvious flaw in my suggestion about the elements.data extractor... That's only ever going to work for the PWI elements.data file, not ESO.

I know that in PWI, in an item struct there is value which gives the positional offset of the icon in the big .dds icon list (or is that only in inventory slots? I can't remember right now). It would possibly make sense that there is perhaps a massive list of all in-game items in memory, however, I don't know where that is in PW, let alone ESO. That might be a path worth investigating though.

i still hafta go look into the suggestions but yes the similar to pw, if not the same except the iconset file ending ends the same ("iconlist_" still there) + text file ofc. I have got the Items + image id from elements.data saved on a mysql database.

The item struct u mean in memory when the game is loaded or a file, and is it based on item id or the name in the txt file? and isnt the dds file just from left to right with the names from txt file? Writing the script to convert the big file in smaller files isnt going to be the problem. I just can't seem to figure out how to connect the surface file to the id.

and just to be clear, afaik the file structure ect is all the same as on PW yes

04/05/2012 20:49 dumbfck#4

Yes, the dds file is just a left to right, top down scan which reflects the (psuedo) surface file names that the client uses. However, I expect for simplicity, the client stores the positional index of each icon when it's loaded into memory so it doesn't have to go into elements.data again to get the surface name or whatever / however it works... sort of like a cache index I suppose. I remember discussing this with Interest07 a while ago and I think we both agreed that this offset is definitely what I suspected it was.

And yes, I mean the item struct in memory when the game is loaded.
However, I just looked in PWI and it doesn't seem that there's a magical huge array of item structs just sitting there unfortunately.
So, this leaves two options that I can think of...

First option would be pretty tough - Figure out the elements.data file structure. I have no idea how the hell Ronny figured that out in sELedit for PWI - all I can think is that it must have taken a damn long time!

Second option... At some point, the client obviously accesses the elements.data file when it loads a new instance of an item. I would guess there's probably a nice function in the client that extracts all the info for a given item, possibly by it's ID or something. It therefore would feasibly be possible to try to track this down, figure out how it works, then write a function injector to recurse the function call with every possible item ID.
Maybe :P

Or... Maybe Interest or Swoosh have a nice little trick up their sleeve :P

I'll have a quick look in PWI to see if I can find such a function as this has peaked my curiosity now and I think it would be a pretty handy little function lol.

04/06/2012 00:53 ItsMeMario1#5

oke so... it seems i might have fcked up the first time i wrote a script to test this file.... cause i was like screw it ima try again and right now "think" i managed to get most the icons (~30 missing each file but they might not be ingame since there are some icons i have never seen in there) but in the data folder there is a file called path.data. containing info. (ID, string lenght, string) reading the iconset txt file line by line and checking if the path.data contains the string it return true for more then 99% of the icons

I still have to check if this is correct but this is what i just got as result

04/06/2012 09:57 amineurin#6

dont know if this helps, i wrote a quick tool to got all images from pwdatabase and save them on my harddrive.
later you have a database.db file and all item images, u can get then name by id from database.db and item image name is item id.

heres the autoit code for the image leecher:

Spoiler

Quote:

;################################################# ######################
; Perfect World Database Extractor v.3
; quick typed by Soulpatrol
;
; How to use:
; 0. there must be a folder named IMG inside the tool folder, for the images
; 1. enter URL from your PW Database (pwi, my, ru) in the gui
; 2. set amount of scanned ID�s , eg. 1000 for ID0 up to ID1000 ID
; 3. press START and go shopping or drink a lot coke while scanning.
;
; when finish scanning, look in Tool Folder for: database.db
; u can open this file with any texteditor, images are stored in IMG folder
;
; entrys looks like:
;[ID][@][NAME]
;6@*Steel Falchion
; u can split Item ID and Item Name with @
;
; have fun :D
;################################################# ####################
#include <INet.au3>
#include <String.au3>
#include <array.au3>
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>

Global $counter,$Start_Stop="stop",$ExitLoop=False,$Split count,$SpanStyle,$NameBuff,$ItemColor

Global $item_color[11]
$item_color[0]="#ffffff;"
$item_color[1]="#8080ff;"
$item_color[2]="#ffdc50;"
$item_color[3]="#aa32ff;"
$item_color[4]="#ff6000;"
$item_color[5]="#ffffff;"
$item_color[6]="#b0b0b0;"
$item_color[7]="#00ffae;"
$item_color[8]="#6cfb4b;"
$item_color[9]="#ff0000;"
$item_color[10]="#80ffff;"

$Form1 = GUICreate("PW Database Extractor", 330, 308, 184, 124)
$Group1 = GUICtrlCreateGroup("Amount", 8, 104, 313, 57)
$Label2 = GUICtrlCreateLabel("Scan up to Item ID:", 16, 128, 97, 17)
$MaxITEMS = GUICtrlCreateInput("35000", 114, 124, 81, 21)
GUICtrlCreateGroup("", -99, -99, 1, 1)
$STATUS = GUICtrlCreateLabel("Ready to scan...", 16, 256, 300, 17)
$Group2 = GUICtrlCreateGroup("Status", 8, 240, 313, 57)
$ITEMCount = GUICtrlCreateLabel("0", 96, 276, 220, 17)
$LabelStatus = GUICtrlCreateLabel("Found Items: ", 16, 276, 77, 17)
GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif")
GUICtrlCreateGroup("", -99, -99, 1, 1)
$Group3 = GUICtrlCreateGroup("Language", 8, 32, 313, 65)
$LabelLang = GUICtrlCreateLabel("Enter: pwi, my or ru", 16, 50, 97, 21)
$URL = GUICtrlCreateInput("pwi", 114, 48, 97, 21)
$CHK_DL = GUICtrlCreateCheckbox("Download Item Images too", 16, 72, 157, 21)
GUICtrlCreateGroup("", -99, -99, 1, 1)
$Group4 = GUICtrlCreateGroup("", 8, 168, 313, 65)
Global $Button_Start = GUICtrlCreateButton("Start Scan", 16, 182, 91, 41, $WS_GROUP)
GUICtrlSetOnEvent($Button_Start, "_start")
Global $Button_Stop = GUICtrlCreateButton("Stop Scan", 126, 182, 91, 41, $WS_GROUP)
GUICtrlSetOnEvent($Button_Stop, "_stop")
Global $Button_Exit = GUICtrlCreateButton("Exit", 236, 182, 75, 41, $WS_GROUP)
GUICtrlSetOnEvent($Button_Exit, "_exit")

GUICtrlCreateGroup("", -99, -99, 1, 1)
$Label4 = GUICtrlCreateLabel('Perfect World Database Extractor by Soulpatrol', 10, 8, 309, 17)
GUICtrlSetFont(-1, 8, 800, 0, "MS Sans Serif")
GUISetState(@SW_SHOW)

While 1
$GUIMsg = GUIGetMsg()
switch $GUIMsg

Case $GUI_EVENT_CLOSE
_exit()
Case $Button_Exit
_exit()

Case $Button_Start
if $ExitLoop=False then
Opt("GUIOnEventMode", 1)
GUISetOnEvent($GUI_EVENT_CLOSE, "_exit")
GUICtrlSetOnEvent($Button_Exit, "_exit")
GUICtrlSetOnEvent($Button_Stop, "_stop")
_start()
endif

Endswitch
WEnd

Func _stop()
GUICtrlSetState($Button_Start,$GUI_ENABLE)
GUICtrlSetState($Button_Stop,$GUI_DISABLE)
$ExitLoop=True
EndFunc
;################################################# ############
FUNC _start()
if $ExitLoop=False then
GUICtrlSetState($Button_Start,$GUI_DISABLE)
GUICtrlSetState($Button_Stop,$GUI_ENABLE)
;GUICtrlSetData($Button_Start, "Stop Scan")
GetDatabaseIDs(GUICtrlRead($MaxITEMS)) ; amount to search

GUICtrlSetData($STATUS, "Finish scanning and new Database saved...")

GUICtrlSetState($Button_Start,$GUI_ENABLE)
GUICtrlSetState($Button_Stop,$GUI_DISABLE)
Opt("GUIOnEventMode", 0)
endif
EndFunc
;################################################# ############
FUNC _exit()
$ExitLoop=True
Exit
EndFunc
;################################################# ############
FUNC GetDatabaseIDs($items)
$Button_Start = False
;open database as unicode
$hDatabase = FileOpen("database.db",34)
;start the loop
$counter=0
For $x=0 To $items-1
$ItemID = $x
;get website content
$sfileHtml = _INetGetSource("http://www.pwdatabase.com/" & GUICtrlRead($URL) & "/items/" & $ItemID)
;remove tab from html text...
$sfileHtml = StringRegExpReplace($sfileHtml , @CRLF, "@")
; get item name
$ItemName=_StringBetween($sfileHtml,'<th class="itemHeader" colspan="2">','@') ; Item Name
IF not @Error=1 then
$Chinese = StringInStr($ItemName[0], "N/") ; chinese string, not ingame item
IF $Chinese = 0 then ;if not found...
formatName($ItemName[0]) ;format the html stuff and split string...
if not $NameBuff="" Then ; if name not empty
FileWriteLine($hDatabase, $ItemID & "@" & $ItemColor & "@" & $NameBuff & @CRLF); save to database
;if dl image checked
if GUICtrlRead($CHK_DL )=1 then
InetGet("http://www.pwdatabase.com/images/icons/generalm/" & $ItemID & ".gif", "img\" & $ItemID & ".gif", 1)
endif
endif
ENDIF
$counter += 1
ENDIF
;set status counter
GUICtrlSetData($ITEMCount, $counter)
GUICtrlSetData($STATUS, "Scanning Item ID: " & $ItemID & "/" & $items)

If $ExitLoop=True Then ExitLoop
next
FileClose($hDatabase)
EndFunc
;################################################# ####################
FUNC formatName($NameTMP)
;look for spanclass
$SpanStyle = StringInStr($NameTMP, "item_color")
IF $SpanStyle > 0 then
;get item color
$ItemColor=_StringBetween($NameTMP,"'","'>")
;format item color
$ColorNum = StringTrimLeft($ItemColor[0], 10)
$ItemColor=$item_color[$ColorNum]
;remove span
$SpecialChar=_StringBetween($NameTMP,'>','<')
$NameBuff=$SpecialChar[0]
Else
$ItemColor= $item_color[0]
$NameBuff=$NameTMP
endif

$NameBuff = StringRegExpReplace($NameBuff ,"●", "�") ; is �
$NameBuff = StringRegExpReplace($NameBuff ,"○", "ο") ; is ο
;look for special html chars...
$pathsplit = StringSplit($NameBuff, ';')
For $Splitcount=1 To $pathsplit[0]-1 ;if found change to unicode...
$SpecialChar=_StringBetween($pathsplit[$Splitcount]&";",'&#',';')
; change special html char to unicode char
$SpecialChar = ChrW($SpecialChar[0])
$NameBuff = StringRegExpReplace($NameBuff, $pathsplit[$Splitcount]&";", $SpecialChar)
next

EndFunc
;################################################# ####################

04/06/2012 11:56 ItsMeMario1#7

@amineurin this would work if only ether saga had a updated database XD but since there are only 2 database websites out there one very nice one but outdated so it doesnt have all the newer items. other one has most of the important items but is missing a lot of info :(

The path.data was the answer and idk what i messed up the first time i tested this file, but i guess i made a typo somewhere in the script back then. Thnxs for the help anyway :)