Home > MalwareAnalysis > Writing Self modifying shellcode

Writing Self modifying shellcode


Nowadays, as I am working on the shellcoding, I thought of writing the self modifying shellcode. Normally, this kind of behaviour is seen in malwares, packers etc, where the code gets modified due to the other instructions. Lets have a look how to do so:


I will be taking the example of MessageBox shellcode. This shellcode I have prepared earlier. Using it as a basis, I would write the self modifying code. The shellcode for MessageBox looks like:


004040A0 31DB       XOR EBX,EBX ;Zero the EBX

004040A2 68 47474700       PUSH 0×00474747 ;PUSH “GGG”

004040A7 8BDC                     MOV EBX,ESP ;Mov stack ptr in EBX

004040A9 50                  PUSH EAX ;Push EAX. We will make it 0 first

004040AA 53               PUSH EBX

004040AB 53             PUSH EBX

004040AC 50            PUSH EAX

004040AD BB 0B05D877        MOV EBX,0x77D8050B ;USER32.MessageBoxA ;

004040B2 FFD3         CALL EBX

004040B4 33C0        XOR EAX,EAX

004040B6 50          PUSH EAX

004040B7 BB A2CA817C         MOV EBX,0x7C81CAA2 ;kernel32.ExitProcess ;

004040BC FFD3                  CALL EBX


This shellcode will produce a messageBox with the title and the message as “GGG”.


So, what is required to write a self modifying code? We can divide the whole complexity in few steps:


  1. We need to know the address in the memory where our code starts execution. This is required to modify the instructions.
  2. We will then change the instructions one by one, modifying the existing instructions and write our shellcode.


The question is how to know the starting address of our code. We will use the following instructions to accomplish this:




These are the FPU instructions. When FSTENV instruction is preceded by some other FPU instruction (in our case, it is FLDZ), then the result of the FSTENV is pushed on to the stack, and the result is none other than the address of the previous FPU instruction. More details can be seen at: http://www.website.masmforum.com/tutorials/fptute/fpuchap3.htm#fstenv


So, what this means, is that if we give these two instructions along with each other in the beginning of our shellcode, we can get the starting point of our code. Lets see how to do it:





The net result of the above two instructions will be the address of FLDZ instruction gets saved on to the stack. So, we have control over the code now.


PS: When the above two instructions are used together, a WAIT instruction is introduced in between, which can be seen when you run it through a debugger.This WAIT is 1 byte instruction. You can also use FNSTENV instruction, which doesnt put an extra WAIT instruction between the two.


Note: The address for the MessageBox function in my windows XP SP2 is 0x77BB050B

and the address for ExitProcess function is 0X7CBBA2CA


The following code does the job:


[Section .text]



global _start



FLDZ ;This takes 2 bytes. However, if you look in debugger, a WAIT instruction is also

;added automatically which takes 1 byte

FSTENV [ESP-0xC] ;So ,now ESP points to the first instruction. Hence, we can control our code ;Also,this instruction takes 5 bytes

POP ECX ;This takes 1 byte instruction.


MOV WORD[ECX],0xDB31 ; Puts xor ebx,ebx by overwriting FLDZ

MOV DWORD[ECX+2],0×47474768 ;Push letter “GGG” on the stack(overwrite the next ;instruction)

MOV BYTE[ECX+6],0×48 ;Placeholder for null byte


MOV BYTE[ECX+6],AL ;changes the string to “GG”,0 and hence an actual string

MOV WORD[ECX+7],0xDC8B ;Mov EBX,ESP– the pointer to the string

MOV BYTE[ECX+9],0×50 ;PUSH EAX, fourth parameter… MB_OK

MOV BYTE[ECX+10],0×53 ;PUSH EBX, the third parameter

MOV BYTE[ECX+11],0×53 ;PUSH EBX, the second parameter

MOV BYTE[ECX+12],0×50 ;PUSH EAX, first parameter… 0

MOV DWORD[ECX+13],0xD8050BBB ;These two instructions move the address of

MOV BYTE[ECX+17],0×77 ;MessageBox function in EBX register

;Note:This will differ for you according to the ;MessageBox function address



MOV BYTE[ECX+22],0×50 ;PUSH EAX (=0) on stack

MOV DWORD[ECX+23],0x81CAA2BB ;These two instructions will

MOV BYTE[ECX+27],0x7c ;call ExitProcess() function. Note: This will differ ;according to the ExitProcess() function address

MOV WORD[ECX+28],0xD3FF ;call EBX

JMP ECX ;Jump to the beginning of our code which has been ;modified


Compile the above code using NASM, and you can use pveReadBin utility (Thanks to corelan team) to extract the hex codes for the above instructions. Test them in the shellcodetest program as follows:



char code[]=
















int main(int argc, char **argv)


int (*func)();





When this program is run, it will pop up the messagebox. I recommend to watch the running executable of the above program in a debugger(my favourites… Immunity/Ollydbg). This will help you to understand all the gaps left behind.



Categories: MalwareAnalysis Tags:
  1. barbeque
    August 1st, 2011 at 17:47 | #1

    nice one b0mbard!!

  1. No trackbacks yet.

Switch to our mobile site