Working with DMA - PCIe Screamer

[AcTiViSioN911]

Hi guys. I have a PCIe Screamer and I am writing a program like Cheat Engine. So far I have implemented search and filtering, but already ran into some questions. I tried to google and figure it out myself, but it didn't give much results. I hope someone can give me a hint and help with a solution.
For example, I took the Terraria game, found a 4 byte number and then repeated it in my program. It took a lot longer, since I either searched all over my memory, or grabbed a lot of unnecessary things, I don't really know. Now in more detail:
I am using MemProcFS with Dokany. I find the game at "M:\name\Terraria.exe-*", there is a 256 TB memory.vmem file. There is a pte.txt file in the memmap folder where the address ranges are listed and I go through each one. Here is the content of this file:

   #    PID    Pages      Range Start-End              FLAGS   Description                                                     
--------------------------------------------------------------------------                                                     
0000   8564        1 00000000001c0000-00000000001c0fff -r-- 32 Terraria.exe
0001   8564        1 00000000001c2000-00000000001c2fff -r-- 32 Terraria.exe
***
0017   8564        1 00000000013c0000-00000000013c0fff -r--
0018   8564        8 00000000013e0000-00000000013e7fff -rw-
***                                                                
006c   8564        1 0000000003870000-0000000003870fff -r-- 32 _DATA-0x3870000.dll
006d   8564        5 0000000003872000-0000000003876fff -r-- 32 _DATA-0x3870000.dll
etc...

I noticed that I can skip lines containing .dll in the Description as they are clearly not related to game memory. However, this is still a lot. For example, a Cheat Engine search for a 4 byte took 1 second and found 884 addresses, ranging from 01A79AD4 to 5923B7BC. My program scanned for 24 seconds (I admit that my methods are not as perfect as those of CE + I use only one thread), but I found 1408 addresses with a range from 001E72A3 to 76555294.
And here my first question is, how does CE find the beginning and end of the memory related to the game, or how does the CE skip everything unnecessary? Probably need to skip 1-2 page / -r-- / -rw- / -rwx ranges, but I'm not sure about that, and I don't want to accidentally skip the addresses I need.
Then I would like to know how to search for pointers to addresses in order to find the correct address after a restart. I think this is also related to my first question. I know how to do this in CE, but have not yet figured out how to implement it myself.
Perhaps someone has already asked similar questions somewhere on the forum, and I would be very grateful for a tip, or at least a hint where to look. I'm not as advanced as the guys on this forum, but I want to learn.
PS: not for sale, purely for my own use.

[elmarcia]

Originally Posted by AcTiViSioN911

Hi guys. I have a PCIe Screamer and I am writing a program like Cheat Engine. So far I have implemented search and filtering, but already ran into some questions. I tried to google and figure it out myself, but it didn't give much results. I hope someone can give me a hint and help with a solution.
For example, I took the Terraria game, found a 4 byte number and then repeated it in my program. It took a lot longer, since I either searched all over my memory, or grabbed a lot of unnecessary things, I don't really know. Now in more detail:
I am using MemProcFS with Dokany. I find the game at "M:\name\Terraria.exe-*", there is a 256 TB memory.vmem file. There is a pte.txt file in the memmap folder where the address ranges are listed and I go through each one. Here is the content of this file:

   #    PID    Pages      Range Start-End              FLAGS   Description                                                     
--------------------------------------------------------------------------                                                     
0000   8564        1 00000000001c0000-00000000001c0fff -r-- 32 Terraria.exe
0001   8564        1 00000000001c2000-00000000001c2fff -r-- 32 Terraria.exe
***
0017   8564        1 00000000013c0000-00000000013c0fff -r--
0018   8564        8 00000000013e0000-00000000013e7fff -rw-
***                                                                
006c   8564        1 0000000003870000-0000000003870fff -r-- 32 _DATA-0x3870000.dll
006d   8564        5 0000000003872000-0000000003876fff -r-- 32 _DATA-0x3870000.dll
etc...

I noticed that I can skip lines containing .dll in the Description as they are clearly not related to game memory. However, this is still a lot. For example, a Cheat Engine search for a 4 byte took 1 second and found 884 addresses, ranging from 01A79AD4 to 5923B7BC. My program scanned for 24 seconds (I admit that my methods are not as perfect as those of CE + I use only one thread), but I found 1408 addresses with a range from 001E72A3 to 76555294.
And here my first question is, how does CE find the beginning and end of the memory related to the game, or how does the CE skip everything unnecessary? Probably need to skip 1-2 page / -r-- / -rw- / -rwx ranges, but I'm not sure about that, and I don't want to accidentally skip the addresses I need.
Then I would like to know how to search for pointers to addresses in order to find the correct address after a restart. I think this is also related to my first question. I know how to do this in CE, but have not yet figured out how to implement it myself.
Perhaps someone has already asked similar questions somewhere on the forum, and I would be very grateful for a tip, or at least a hint where to look. I'm not as advanced as the guys on this forum, but I want to learn.
PS: not for sale, purely for my own use.

Cheat engine filter writable/readable memory regions to speed up the process and use multithreading for scans so its even faster.

Here is the logic it follows:

1) Query Mem regions and filter valid ones (store in a struct (baseAddress, size))
For each valid region increment totalMemorySize
2) Obtain scan block size (totalMemorySize / threadCount), last block will be a bit longer, so don't forget to append rest.
3) Make scan threads structs
3.a) Read process memory is super slow so don't call it on a loop for reading lets say (4 bytes), call it to read a fixed buffer size, (CE uses 1024*4 buffer size for dword scans i think) then perform your "mini scan" in that buffer.
3.b) Make scan functions that works for you, when searching for aligned memory you will skip a lot but scan speed is increased.
4) Start and wait for scan threads to end.


1)

 Spoiler

  //VIRTUAL QUERY REGIONS
  while (Virtualqueryex(processhandle,pointer(currentBaseAddress),mbi,sizeof(mbi))<>0) and (currentBaseAddress<stopaddress) and ((currentBaseAddress+mbi.RegionSize)>currentBaseAddress) do  
  begin

    begin
      if PtrUint(mbi.BaseAddress)<startaddress then
      begin
        dec(mbi.RegionSize, startaddress-PtrUint(mbi.BaseAddress));
        mbi.BaseAddress:=pointer(startaddress);
      end;

      if PtrUint(mbi.BaseAddress)+mbi.RegionSize>=stopaddress then
        mbi.RegionSize:=stopaddress-PtrUint(mbi.BaseAddress);

      // FLAGS TO FILTER REGIONS
      validRegion:=(mbi.State=mem_commit);
      validRegion:=validregion and (PtrUint(mbi.BaseAddress)<stopaddress);
      validregion:=validregion and ((mbi.Protect and page_guard)=0);
      validregion:=validregion and ((mbi.protect and page_noaccess)=0);
      validRegion:=validRegion and (not (not scan_mem_private and (mbi._type=mem_private)));
      validRegion:=validregion and (not (not scan_mem_image and (mbi._type=mem_image)));
      validRegion:=validregion and (not (not scan_mem_mapped and (mbi._type=mem_mapped)));
      validRegion:=validregion and (not (Skip_PAGE_NOCACHE and ((mbi._type and PAGE_NOCACHE)>0)));
      validRegion:=validregion and (not (Skip_PAGE_WRITECOMBINE and ((mbi._type and PAGE_WRITECOMBINE)>0)));

      if validregion then
      begin
        //initial check passed, check the other protection flags to see if it should be scanned

        //fill in isWritable, isExecutable, isCopyOnWrite: boolean;
        isWritable:=((mbi.protect and PAGE_READWRITE)>0) or
                    ((mbi.protect and PAGE_WRITECOPY)>0) or //writecopy IS writable
                    ((mbi.protect and PAGE_EXECUTE_READWRITE)>0) or
                    ((mbi.protect and PAGE_EXECUTE_WRITECOPY)>0);

        isExecutable:=((mbi.protect and PAGE_EXECUTE)>0) or
                      ((mbi.protect and PAGE_EXECUTE_READ)>0) or
                      ((mbi.protect and PAGE_EXECUTE_READWRITE)>0) or
                      ((mbi.protect and PAGE_EXECUTE_WRITECOPY)>0);

        isCopyOnWrite:=((mbi.protect and PAGE_WRITECOPY)>0) or
                       ((mbi.protect and PAGE_EXECUTE_WRITECOPY)>0);

       
        isdirty:=(mbi.protect and PAGE_DIRTY)>0;


        // scan filter options
        case scanWritable of
          scanInclude: validregion:=validregion and isWritable;
          scanExclude: validregion:=validregion and (not isWritable);
        end;

        case scanExecutable of
          scanInclude: validregion:=validregion and isExecutable;
          scanExclude: validregion:=validregion and (not isExecutable);
        end;

        case scanCopyOnWrite of
          scanInclude: validregion:=validregion and isCopyOnWrite;
          scanExclude: validregion:=validregion and (not isCopyOnWrite);
        end;

        {$ifdef darwin}
        case scanDirty of
          scanInclude: validregion:=validregion and isDirty;
          scanExclude: validregion:=validregion and (not isDirty);
        end;
        {$endif}
      end;

      if not validregion then
      begin
        // ignore region 
        currentBaseAddress:=PtrUint(mbi.BaseAddress)+mbi.RegionSize;

        continue;
      end;


      //still here, so valid

        // store region data
        memRegion[memRegionPos].BaseAddress:=PtrUint(mbi.baseaddress);  //just remember this location
        memRegion[memRegionPos].MemorySize:=mbi.RegionSize;
        memRegion[memRegionPos].startaddress:=pointer(ptrUint(totalProcessMemorySize)); //starts from 0, for unknown scans

        inc(memRegionPos);
        if (memRegionPos mod 16)=0 then // reallocate if needed
          setlength(memRegion,length(memRegion)+16);
   
      inc(totalProcessMemorySize,mbi.RegionSize); //add this size to the total


    end;


    currentBaseAddress:=PtrUint(mbi.baseaddress)+mbi.RegionSize;

  end;

2)

 Spoiler

  blocksize:=totaladdresses div threadcount;

3)

 Spoiler

 scannersCS.Enter; //block access by the mainthread on the scanners object, could scanner[14] has not yet been created when doing a progress request
  try
    setlength(scanners,threadcount);
    j:=0; //start at memregion 0
    leftfromprevious:=0;
    offsetincurrentregion:=0;

    for i:=0 to threadcount-1 do
    begin
    //  OutputDebugString(format('Creating scanner %d',[i]));
      scanners[i]:=tscanner.Create(true, OwningMemScan.ScanresultFolder);
      scanners[i].scannernr:=i;
      scanners[i].OwningScanController:=self;


      scanners[i]._startregion:=j;
      scanners[i].startaddress:=memRegion[j].BaseAddress+offsetincurrentregion;

      //scanners[i].maxregionsize:=0; //Original Code, no longer needed
      currentblocksize:=0;
      inc(currentblocksize,memregion[j].MemorySize-offsetincurrentregion);
      scanners[i].maxregionsize:=currentblocksize;


      if i=(threadcount-1) then
      begin
        //if it's the last thread, just give it what's left
        scanners[i].stopaddress:=stopaddress;
        scanners[i]._stopregion:=memregionpos-1;

        //define maxregionsize , go from current till end (since it'll scan everything that's left)
        while j<memregionpos do
        begin
          if scanners[i].maxregionsize<memregion[j].MemorySize then
            scanners[i].maxregionsize:=memregion[j].MemorySize;

          inc(j);
        end;
      end
      else
      begin
        //not the last thread
        inc(j);

        while (currentblocksize<blocksize) and (j<memregionpos) do
        begin
          if scanOption<>soUnknownValue then //not a unknown initial value scan, so it doesn't need overlap
          begin
            if scanners[i].maxregionsize<memregion[j].MemorySize then
              scanners[i].maxregionsize:=memregion[j].MemorySize;
          end;

          inc(currentblocksize,memregion[j].MemorySize);
          inc(j);
        end;
        dec(j);

        if scanners[i].maxregionsize=0 then //(currentblocksize<blocksize)
          scanners[i].maxregionsize:=memregion[j].MemorySize;

        scanners[i]._stopregion:=j;
        scanners[i].stopaddress:=(memregion[j].BaseAddress+memregion[j].MemorySize);

        //take off the bytes that are too many for this block
        leftfromprevious:=currentblocksize-blocksize;
        dec(scanners[i].stopaddress,leftfromprevious);

        if leftfromprevious=0 then
        begin
          inc(j); //nothing left in this region
          offsetincurrentregion:=0;
        end else offsetincurrentregion:=memregion[j].MemorySize-leftfromprevious;


      end;
    //  OutputDebugString(format('startregion = %d',[scanners[i]._startregion]));
    //  OutputDebugString(format('stopregion = %d',[scanners[i]._stopregion]));
    //  OutputDebugString(format('startaddress = %x',[scanners[i].startaddress]));
    //  OutputDebugString(format('stopaddress = %x',[scanners[i].stopaddress]));

    //  OutputDebugString(format('j = %d',[j]));
    //  OutputDebugString(format('leftfromprevious = %x',[leftfromprevious]));
    //  OutputDebugString(format('offsetincurrentregion = %x',[offsetincurrentregion]));



      if scanners[i].maxregionsize>buffersize then
        scanners[i].maxregionsize:=buffersize;

   //   OutputDebug String(format('maxregionsize = %x',[scanners[i].maxregionsize]));
              

      //now configure the scanner thread with the same info this thread got, with some extra info
      scanners[i].compareToSavedScan:=compareToSavedScan;
      scanners[i].savedscanname:=savedscanname;
      scanners[i].scanType:=scanType; //stFirstScan obviously
      scanners[i].scanoption:=scanoption;
      scanners[i].variableType:=VariableType;
      scanners[i].customType:=CustomType;
      scanners[i].roundingtype:=roundingtype;
      scanners[i].scanValue1:=scanvalue1; //usual scanvalue
      scanners[i].scanValue2:=scanValue2; //2nd value for between scan
      scanners[i].unicode:=unicode;
      scanners[i].OnlyOne:=OnlyOne or isUnique;
      scanners[i].caseSensitive:=caseSensitive;
      scanners[i].percentage:=percentage;
      scanners[i].hexadecimal:=hexadecimal;
      scanners[i].binaryStringAsDecimal:=binaryStringAsDecimal;

      scanners[i].fastscanalignsize:=fastscanalignsize;
      scanners[i].fastscanmethod:=fastscanmethod;
      scanners[i].fastscandigitcount:=fastscandigitcount;
      scanners[i].variablesize:=variablesize;
      scanners[i].floatscanWithoutExponents:=floatscanWithoutExponents;
      scanners[i].inverseScan:=inverseScan;
      scanners[i].luaformula:=luaformula;
      scanners[i].newluastate:=newluastate;



      if i=0 then //first thread gets the header part
      begin
        if scanoption=soUnknownValue then
          datatype:='REGION'
        else
          datatype:='NORMAL';

        scanners[i].AddressFile.WriteBuffer(datatype,sizeof(datatype));

      end;

    end;
  finally
    scannersCS.Leave;
  end;

3.a)

 Spoiler

for i:=startregion to stopregion do
    begin
      if terminated or OwningScanController.Terminated then exit;
      if i=startregion then
      begin
        currentbase:=startaddress;
        toread:=OwningScanController.memregion[i].MemorySize-(startaddress-OwningScanController.memregion[i].BaseAddress);

        //set oldbuffer and point to the exact start
        oldbuffer:=OwningScanController.memregion[i].startaddress;
        inc(oldbuffer, startaddress-OwningScanController.memregion[i].BaseAddress);
      end
      else
      begin
        currentbase:=OwningScanController.memregion[i].BaseAddress;
        toread:=OwningScanController.memregion[i].MemorySize;

        oldbuffer:=OwningScanController.memregion[i].startaddress;
      end;

      if (i=stopregion) and ((currentbase+toread)>stopaddress) then
        toread:=stopaddress-currentbase;

      //also try to read the last few bytes and add variablesize if needed
      if (currentbase+toread)<(OwningScanController.memregion[i].BaseAddress+OwningScanController.memregion[i].MemorySize-variablesize) then
        inc(toread, variablesize-1);

      repeat
        size:=toread;
        if (size>buffersize) then size:=buffersize;

        actualread:=0;

        if size<toread then //there's something left to scan, so I can add the variablesize to it
          ReadProcessMemory(phandle,pointer(currentbase),memorybuffer,size+variablesize-1,actualread)
        else
          ReadProcessMemory(phandle,pointer(currentbase),memorybuffer,size,actualread);


        lastpart:=302;

        firstnextscanmem(currentbase,memorybuffer,oldbuffer,actualread);

        lastpart:=398;

        inc(scanned,size); //for the progressbar
        dec(toread,size);
        inc(oldbuffer,size);
        inc(currentbase,size);

      until toread=0;

3.b)

 Spoiler

function TGroupData.DWordScan(value: dword; buf: pointer; var startoffset: integer): boolean;
var current: pointer;
  i: integer;
  align: integer;
begin
  result:=false;
  if outoforder_aligned then
    align:=4
  else
    align:=1;

  current:=buf;
  inc(current, startoffset);
  i:=startoffset;

  while i<blocksize-3 do
  begin
    if pdword(current)^=value then
    begin
      startoffset:=i+1;
      result:=true;
      exit;
    end;

    inc(current,align);
    inc(i,align);
  end;
end;

4)

 Spoiler

  for i:=0 to length(scanners)-1 do
    scanners[i].start;

  //prepare the result files
  try

    OwningMemScan.found:=0;


    //and now we wait
    for i:=0 to threadcount-1 do
    begin
      while not (terminated or scanners[i].isdone) do
      begin
{$ifdef windows}
        WaitForSingleObject(scanners[i].Handle,25); //25ms, an eternity for a cpu

        if (OwningMemScan.progressbar<>nil) or (assigned(owningmemscan.OnGuiUpdate)) then
          synchronize(updategui);
{$else}
        sleep(25)
{$endif}

      end;

You can look at CE code its a bit messy but better than nothing

[AcTiViSioN911]

Originally Posted by elmarcia

Cheat engine filter writable/readable memory regions to speed up the process and use multithreading for scans so its even faster.

Thanks a lot for the tips, I really have a lot to change. Apart from optimization, I'm stuck on the question of pointers and offsets. In the example of the game Terraria, I searched for a static address using CE, but I never found it. Instead, I found something similar, with the start of + 6 offsets to the address I needed. This THREADSTACK0 is constantly changing (only CE can work with it) and I have no idea how to find it. It looks like I'm missing something, but I reviewed many CE guides and did not find an explanation of what to do if a static address such as Terraria.exe + ** cannot be found. Perhaps this game is more complicated than the one used in the examples and where in 2 scans it was possible to find a static address ... I cannot use debuggers or any other programs, only the functionality that pcileech + memprocfs gives. I heard that it is possible to use WinDBG, but only a small amount of functionality without breakpoints, and I'm not sure if that will help me with anything. Also, I don't quite understand how to find offsets, because as I understand it, CE uses debugger with breakpoints, but I can't. So far, I manually find the addresses I need and try to identify a pattern by which I can repeat the search like a sequence of bytes before the start of the part I need), but I think this is not the best method. If you have any ideas, please share with me.

[AcTiViSioN911]

I am almost satisfied with the performance of the program, and most likely this is the maximum DMA speed. The question of finding pointers and offsets is still open...

[iflores]

Why not just use Cheat Engine with PCILeech directly? There's no need to write a new Cheat Engine.

[miserymodz]

sorry to bump this threat. the pcileech is down and just wondering if you ever made what you were working on or what

[StrawGuy]

Originally Posted by miserymodz

sorry to bump this threat. the pcileech is down and just wondering if you ever made what you were working on or what

I'm exactly on the same boat as you. So far I've found these;



They both "work" for me. But the speed of searching a value is below par even though my DMA speed is over 180MB/s

[ltMorlock]

Originally Posted by StrawGuy

I'm exactly on the same boat as you. So far I've found these;



They both "work" for me. But the speed of searching a value is below par even though my DMA speed is over 180MB/s

Hey maybe you can help me ?
i installed server from the Rep. and copied pcileech.dll in the folder. When i want to start server with connectet PCISquirrell card i get an error:
[22:27:10.591] ServerMain: Initializing PCILeech...
java.lang.RuntimeException: Unable to initialize PCILeech.
at iflores.ceserver.pcileech.ServerMain.main(ServerMa in.java:71)
*** Server died with exit code -1
Mayber you can help me?

Originally Posted by AcTiViSioN911

I am almost satisfied with the performance of the program, and most likely this is the maximum DMA speed. The question of finding pointers and offsets is still open...

Awesome... did you plan a release for this?