.gtx files are wrapped images of different image formats - not just dds. Most often they are either DDS or TGA. If you are looking for icons for gear and accessories, they are almost always .tgs files. DDS are used for shader models and textures.
There are different methods to unwrap each of these and you need to find out which method to use by examining the other bytes of the header. One method involves deleting bytes (TGA) the other replacing the "GEO " with "DDS " and not deleting it. Use these resources for information about the DDS and TGA formats:
(Targa -- TGA)
(DDS --- version as used in Rohan. Note this immediately follows the file signature "DDS ")
The easiest way to detect which format to use is to look at the bytes at 0-based offsets 4 and 5 (the two bytes immediately following the "GEO " file signature). In a DDS file these 2 bytes will always have a value since they are the low order 2 bytes if the image length in little-endian binary format. It is very very rare in huge .dds files that these two bytes will be zero. In TGA files the first byte is the length of the header's description field (always zero for ROHAN). The next byte is the color map type. Again in always 0 in ROHAN.
If both those bytes are 0, you are working with a TGA file disguised as a .GTX file. If either byte 4 or byte 5 has a value, it is DDS.