Author Archives: alr
slithering along a file with python
The ‘file’ command is a nice tool. It has a database of filetypes and “magic” numbers which correspond to offsets and values within a file and are used to hazard a guess as to what type of file it is. On my system, the /usr/share/file/magic database has 13474 lines in it. Quite a bit of knowledge about filetypes at your fingertips!
To use it simply:
$ file <targetfile>
Example:
$ file /pictures/nice.jpg
/pictures/nice.jpg: JPEG image data, JFIF standard 1.02or
$ file ./unknown
./unknown: VMS Alpha executableWhat happens when dealing with “unknown” file types that may not be accurately described by the “file” command’s knowledge of filetypes? Or, what happens when a file contains many other files within it that we can easily get to? We can attempt to peer inside an unknown container file and find what types of other files it is made of… by sliding along the file and comparing every offset to the magic database.
Luckily, there is a python binding the “magic” database.
# apt-get install python-magic
And a handy example is included in /usr/share/doc/python-magic/examples/example.py.
Excellent. This is just what we need. Our algorithm is simple. Loop over each offset in the file and see what python-magic thinks it is. Interesting offsets can then be identified and extracted for further analysis.
Here’s a quick one-off python script to do just that:
magicslide.py:
# !/usr/bin/env python
"""
%s
will be checked at each offset to see what the magic offset
database from the "file" command's database thinks it is.
Entries that return 'data' will be filtered because they are boring.
"""
import magic
import os
import sys
def usage():
sys.stdout.write( __doc__ % os.path.basename(sys.argv[0]))
sys.exit(0)
def analyze(ms,buffer):
return ms.buffer(buffer)
def output(offset,s):
sys.stdout.write("%08x:%s\n" % (offset,s) )
try:
filename = sys.argv[1]
except:
usage()
try:
f = open(filename)
except:
sys.stderr.write("could not open %s\n" % filename)
sys.exit(1)
filedata = f.read()
totallen = len(filedata)
buffsize = 4096 # a nice big chunk of file
# load the magic db
ms = magic.open(magic.MAGIC_NONE)
ms.load()
for offset in range(0,totallen):
end_offset = min(offset+buffsize+1,totallen)
kind = analyze ( ms, filedata[offset:end_offset] )
if kind != 'data':
output( offset, kind )Sample output looks like:
0001047c:Hitachi SH big-endian COFF executable, not stripped
00010493:PCX ver. 2.5 image data
000104a8:MIPSEB MIPS-III ECOFF executable not stripped - version 255.26
000104b2:\012- 8086 relocatable (Microsoft)
000104b8:PCX ver. 2.5 image data
000104bd:MPEG ADTS, layer I, v1, 32 kBits, 32 kHz, Monaural
000104c1:MPEG ADTS, layer I, v1, 448 kBits, 32 kHz, Stereo
000104c8:DBase 3 data file
000104cc:LANalyzer capture file
000104e0:PCX ver. 2.5 image data
000104e8:shell archive or script for antique kernel text
000104ef:PCX ver. 2.5 image data
000104f6:MPEG-4 LOAS
00010508:AmigaOS bitmap font
0001050c:PCX ver. 2.5 image data
00010514:shell archive or script for antique kernel text
0001051c:MIPSEB MIPS-III ECOFF executable not stripped - version 0.10
00010522:MPEG-4 LOAS
00010530:Hitachi SH big-endian COFF executable, stripped
00010538:DBase 3 data file
0001053c:PCX ver. 2.5 image data
00010544:shell archive or script for antique kernel text
00010549:MPEG ADTS, layer I, v1, 32 kBits, 32 kHz, Stereo
00010560:DBase 3 data file
Well, it’s still pretty messy and the data may be wrong, but it’s more than we had to go on before for our analysis of this unknown file type. There are obvious false positives here, but things like images such as JPGs, PNGs, etc. can probably be readily identified in the file of interest.
# aa
openmoko: cool little linux box
The OpenMoko project ( http://www.openmoko.org ) has “freed” the cell phone. OpenMoko is an open development platform with complete hardware specs (as complete as possible) that runs linux, can be recompiled from scratch from source code, and operates as a normal “unlocked” cellular device. This news isn’t new, but it is the first time I’m writing about it. The openmoko team actually released their second version of the cellphone hardware earlier this month (called GTA02 but nothing to do with the video game) with some significant new features including WiFi and accelerometers.
If you are like me, then you remember seeing the word “linux” in the hallowed directory listings of ftp.cdrom.com circa 1994 and thinking… hey what’s this new word? A few hours/days later, after borrowing a laptop from the school A/V department, getting comfy trashing the existing operating system fdisk style and loading slackware from a lot of floppy disks, you were greeted by a fully-bootable operating system that measured its speed in BogoMips and could do most of the things the computers in the Sun lab could do except that you were root (legitimately).
So now we’ve had Linux for a while, its used all over the place and is a system that people seem to have gotten pretty comfortable with. This level of ease and comfort is now available in the form of “the device you take with you everywhere” …your cellphone is now just a little linux box. Why is this cool? Because now I can talk to my friends, and ssh into my server from my cell phone (or vice versa). Oh yeah, and do all that other stuff that Linux does, like run Apache, FTP, NFS, torrent, or scan your systems with Nessus (theoretically).
The OpenMoko project has already suffered/gained from the normal Linux way of things and there are a few different distributions available. Developers being the way they are have splintered off from the official OpenMoko distribution and created their own distros already. One in particular, an “Underground” distro has even gone so far as to scrap X11 for windowing and use the framebuffer directly. The wheel gets reinvented once again. Hopefully this time with built-in battery powered spinners.
There are numerous ways this little toy could be used for security testers. Since it has both WiFi and can use the GSM networks (AT&T and T-Mobile work ok in the states), this would make a nice little remote access device. All you need to do is leave it in the proximity of a location with WiFi then dial in (pppd) from across the world or anywhere cellular data connections can go (if you don’t like the idea of being in physical proximity of your targets or aren’t good at talking to beefy security guards who wonder why your laptop is beeping.) Alternatively, since it has USB, plug into a corporate computer, then dial in from the cellular side and route through newly-befriended corporate system. The possibilities here are numerous. GPS-activated, bluetooth aware, motiondetecting wifi gprs connection machine…
All in all, a cool device. Stay tuned for fun stuff to do with it.
- theOtherAaron
