tag:blogger.com,1999:blog-14046602049664540422018-02-08T08:32:18.610-08:00Meme OverMostly computer security, Internet culture, and information warfare. Mostly.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.comBlogger53125tag:blogger.com,1999:blog-1404660204966454042.post-69318177380677081502015-09-27T21:59:00.002-07:002015-09-27T22:54:36.194-07:00KPROCESS - InstrumentationCallback - Get callbacks on return from kernel modeLong time no see everybody!<br /><br />I was pointed at a really interesting article this week and thought I'd share.<br /><br />It turns out in Windows Vista x64 and later you can ask the kernel to invoke a callback on transition out of kernel mode. KPROCESS has a field called InstrumentationCallback that you can set with a call to NtSetInformationProcess with a ProcessInfoClass of 0x28 and a InputBuffer that contains the pointer to your callback.<br /><br />Advantages:<br /><br /><ul><li>User mode only, no driver or kernel debugger required</li><li>Affects the entire process (including injected threads)</li></ul><div><br /></div>Disadvantages:<br /><br /><ul><li>Windows x64 only</li><li>Apparently doesn't work on WOW64 (though maybe some jiggery-pokery could get it done)</li><li>Required Dr7 to be set in most cases so not great at catching malicious actors</li></ul><br />The original article:&nbsp;<a href="http://www.codeproject.com/Articles/543542/Windows-x-system-service-hooks-and-advanced-debu">Windows x64 system service hooks and advanced debugging</a>&nbsp;and check out the author's blog&nbsp;<a href="http://everdox.blogspot.com/">http://everdox.blogspot.com/</a>&nbsp;for other interesting RE posts.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-33133503045809839242013-11-05T21:15:00.000-08:002013-11-05T21:38:47.080-08:00Authenticode and Antivirus Detection RevisitedIt's time to revisit code signing and antivirus detection! Two years ago I looked into whether or not Authenticode signatures (Microsoft object code signing for PE files) influenced the decisions of antivirus engines.<br /><br />In the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection.html">first part</a>&nbsp;I described the process of finding, signing and testing some malware with <a href="http://www.virustotal.com/">VirusTotal</a>&nbsp;and it appeared that adding an Authenticode signature to a known piece of malware drastically lowered its detection rate. After an astute observation in the comments and some thinking about it, I decided this was more likely due to the fragility of the signatures created by antivirus vendors. In the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection_08.html">second part</a>&nbsp;I test this theory by using the VirusTotal API to test 100 malware samples to compare the response of adding an Authenticode signature to changing the binary in other ways. The result largely confirmed that rather than code signing that was defeating the antivirus scans, it was that the binary was changing at all.<br /><br />These results were ultimately unsatisfying, partly because of the surprising fragility of the antivirus signatures but also because of the test methodology. VirusTotal is a wonderful resource but is only testing the core antivirus engine of these security products which these days is only a small part of the protective services provided by security software. Additionally, the certificate that the malware was signed with was not endorsed by any certificate authority and was not deemed trusted by the operating system.<br /><br />This time around I investigated how the actual security products responded, and used a trusted certificated to sign the malware. Unlike last time, the number of engines tested is much lower and the number of samples used is only one. Turns out that this kind of testing is very time intensive if you don't already have the infrastructure set up. While I did have some free time during the US Government shutdown while waiting for a US visa, there's a limit to how many virtual machines I felt like setting up.<br /><br /><b>Part 1 - The Malware</b><br />This time the malware sample used is VPN-Pro.exe (MD5:&nbsp;8eda7dfa4ec4ac975bb12d2a3186bbeb)&nbsp;as contributed by the redoubtable&nbsp;<a href="https://twitter.com/headhntr">@headhntr</a>&nbsp;to the&nbsp;<a href="http://syrianmalware.com/">Syrian Malware Samples Project</a>. As described by the <a href="https://citizenlab.org/2013/06/a-call-to-harm/">Citizenlab analysis</a>&nbsp;it is a trojanized version of Freegate 7.35 written using .NET 3.5 that drops the ShadowTech RAT. The campaign was targeted at dissidents in Syria, make sure that you check out the analysis, it's fascinating but a little outside the scope of this post.<br /><br />When Citizenlab submitted VPN-Pro.exe to VirusTotal in June 2013 it was detected by 5/46 antivirus engines. When I checked in October the <a href="https://www.virustotal.com/en/file/829e137279f691e493c211108b62c8e15b079bd619ba19ad388450878e0585d0/analysis/">VirusTotal report</a>&nbsp;had been updated to show detection by 34/47 antivirus engines.<br /><br /><b>Part 2 - The Antivirus Suites</b><br />The test environment is a series of Windows 8.1 Virtual Machines with the following security software suites installed.<br /><br /><ul><li>Sophos Endpoint Client Protection</li><li>McAfee All Access</li><li>Norton 360</li><li>Windows Defender (as installed by default with Windows 8.1)</li></ul><div>In addition Chrome was installed in each VM, and each sample was submitted to VirusTotal.</div><div><br /></div><div><b>Part 3 - The Transformations</b></div><div>Six versions of the sample were tested, the original and five transformed versions.</div><div><br /></div><div><u>1. Padded</u></div><div>VPN-Pro.exe with 1024 'A' characters appended to the end.</div><div>SHA256:&nbsp;<a href="https://www.virustotal.com/en/file/54d9f5767ec3a7aba6754dacc998d57bb54e793750f2e5b1e63e37cc9c43da6e/analysis/1382678003/">54d9f5767ec3a7aba6754dacc998d57bb54e793750f2e5b1e63e37cc9c43da6e</a></div><div><br /></div><div><u>2. Random Padding</u></div><div>VPN-Pro.exe with random bytes added to match the length of the signed version.</div><div>SHA256:&nbsp;<a href="https://www.virustotal.com/en/file/8428aa9dfa69438d98b0008b0dc7c9e8135889d893a77d5536aacf8b7e1ad6e7/analysis/1382679731/">8428aa9dfa69438d98b0008b0dc7c9e8135889d893a77d5536aacf8b7e1ad6e7</a></div><div><br /></div><div><u>3. Authenticode (Self-signed certificate)</u></div><div>VPN-Pro.exe signed with a test certificate (as describe in part 1 of this series)</div><div>SHA256:&nbsp;<a href="https://www.virustotal.com/en/file/f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7/analysis/1382678288/">f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7</a></div><div><br /></div><div><u>4. Damaged Authenticode (Self-signed certificate)</u></div><div>VPN-Pro.exe as prepared in number 3 but with roughly 10% of the signature bytes replaced randomly.</div><div>SHA256:&nbsp;<a href="https://www.virustotal.com/en/file/1ddf2de1bb8d289b6b77843bc2b9a685d31d0f17371691e3f4b81faa383c7769/analysis/1382678303/">1ddf2de1bb8d289b6b77843bc2b9a685d31d0f17371691e3f4b81faa383c7769 &nbsp;</a></div><div><br /></div><div><u>5. Authenticode (Trusted certificate)</u></div><div>VPN-Pro.exe signed with a trusted code signing certificate from <a href="http://startssl.com/">StartSSL.com</a>&nbsp;(thanks to the anonymous benefactor that helped me with this part).</div><div>SHA256:&nbsp;<a href="https://www.virustotal.com/en/file/d9deeaa7762072d5cb8f99ecea7c1acf32354ba4486f9afb01f4404149b919fd/analysis/1383538841/">f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7</a></div><div><br /></div><div><b>Part 4 - The Method</b></div><div>Each VM had the trial version (as found on the vendor's website) of the security software installed. Windows and the security software were then updated. Next Chrome was used to download each sample and the reactions of the browser and security software were recorded. If there was no reaction from the security software or browser a manual scan was initiated where possible. In the case of Windows Defender the same tests were undertaken using Internet Explorer as well as Chrome.</div><div><br /></div><div><b>Part 5 - The Results</b><br /><u><br /></u><u>Summary</u></div><div><table cellpadding="0" cellspacing="0" dir="ltr" style="font-family: arial,sans,sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="120"></col><col width="78"></col><col width="78"></col><col width="77"></col><col width="77"></col><col width="75"></col><col width="75"></col></colgroup><tbody><tr style="height: 17px;"><td style="border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Original</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Padded</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Random Padding</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Damaged Signature</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Self Signed</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Trusted Signature</td></tr><tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">McAfee All Access</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Norton 360</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Sophos Endpoint</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: lime; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Windows Defender</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: lime; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"><b><span style="color: magenta;">Mild Warn</span></b></td></tr><tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Virus Total</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">34/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/46</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">8/47</td></tr><tr style="height: 17px;"><td style="border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="background-color: lime; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="2" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Nothing detected</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Warning, this may harm your PC</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Reputation based detection</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr><tr style="height: 17px;"><td style="background-color: red; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Explicitly marked as virus</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr></tbody></table></div><div><u><br /></u><u>McAfee All Access</u></div><div><u><br /></u></div><div>The original sample was detected as malware and automatically removed.</div><div class="separator" style="clear: both; text-align: center;"><a href="http://3.bp.blogspot.com/-uccKFviTaYY/Unm_YcS-NaI/AAAAAAAAfx4/Qn2Gu7JVJHA/s1600/mcafee-vpn-pro.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="271" src="http://3.bp.blogspot.com/-uccKFviTaYY/Unm_YcS-NaI/AAAAAAAAfx4/Qn2Gu7JVJHA/s400/mcafee-vpn-pro.exe.png" width="400" /></a></div><div><br /></div><div>All other samples were detected as malware and quarantined.</div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-zJkmQ7MB5Fs/Unm_4E56hsI/AAAAAAAAfyA/D3EOhrFml_w/s1600/mcafee-vpn-pro.signedrm.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="http://1.bp.blogspot.com/-zJkmQ7MB5Fs/Unm_4E56hsI/AAAAAAAAfyA/D3EOhrFml_w/s400/mcafee-vpn-pro.signedrm.exe.png" width="400" /></a></div><div><br /></div><div>Considering that on the VirusTotal scan McAfee did not detect any of the transformed samples, my guess is that the Quarantined dialog is shown on heuristic or binary reputation based matches while the automated removal dialog is for signature based matches.</div><div><br /></div><div><u>Norton 360</u></div><div><br /></div><div>The original sample was detected as malware and automatically removed.</div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-1KkS8SlpNKY/UnnAkear3QI/AAAAAAAAfyI/rumPatwrz4s/s1600/norton-vpn-pro-1.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="459" src="http://1.bp.blogspot.com/-1KkS8SlpNKY/UnnAkear3QI/AAAAAAAAfyI/rumPatwrz4s/s640/norton-vpn-pro-1.exe.png" width="640" /></a></div><div><br /></div><div>The transformed samples were also all detected and automatically removed.</div><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-LFElhI6CM0Q/UnnA0dB3MxI/AAAAAAAAfyQ/CYebGU_iyyM/s1600/norton-vpn-pro.signedrm-1.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="459" src="http://2.bp.blogspot.com/-LFElhI6CM0Q/UnnA0dB3MxI/AAAAAAAAfyQ/CYebGU_iyyM/s640/norton-vpn-pro.signedrm-1.exe.png" width="640" /></a></div><div>The difference being that all the transformed files were detected as the threat "WS.Reputation.1" and the Threat Type of "Insight Network Threat". This suggests to me that the cloud binary reputation service is flagging these files as harmful largely because they have note been seen before. Again, the Symantec engine did not detect the transformed files during the VirusTotal submission (I assume that Norton 360 uses the Symantec antivirus engine).</div><div><br /></div><div><u>Sophos Client Endpoint Protection</u></div><div><u><br /></u></div><div>Again, the original sample was detected and removed.</div><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-IVHOvxUAmcE/UnnCbq7YwpI/AAAAAAAAfyg/xy-GBuJ562s/s1600/sophos-chrome-vpn-pro.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="512" src="http://2.bp.blogspot.com/-IVHOvxUAmcE/UnnCbq7YwpI/AAAAAAAAfyg/xy-GBuJ562s/s640/sophos-chrome-vpn-pro.exe.png" width="640" /></a></div><div><br /></div><div>In fact all the transformations (except one) of VPN-Pro.exe were detected in the same way as the original and were tagged as Mal/Generic-S. The original was flagged as Mal/Generic-S on VirusTotal as well but the transformations weren't likewise flagged at the time, unsure whether this is due to some fuzzy matching or updated signatures.</div><div><br /></div><div>However, the Authenticode version with the trusted certificate was downloaded without complaint. Considering that the self signed version was flagged as malicious, I'm drawn to conclude that the validity of the signature was taken into account.</div><div class="separator" style="clear: both; text-align: center;"></div><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-FC1LaC6sm8s/UnnDjhTwVFI/AAAAAAAAfyw/kovFdtxASkY/s1600/sophos-vpn-pro.signedrm.exe.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="360" src="http://4.bp.blogspot.com/-FC1LaC6sm8s/UnnDjhTwVFI/AAAAAAAAfyw/kovFdtxASkY/s640/sophos-vpn-pro.signedrm.exe.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">A manual scan was run after the initial download completed.</td></tr></tbody></table><div><u><br /></u></div><div><u>Windows Defender</u></div><div><u><br /></u></div><div>Windows Defender on Internet Explorer gave the largest variety of messages, here's all six:</div><div class="separator" style="clear: both; text-align: center;"></div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-o4WeD6g3IuU/UnnFPSCSVnI/AAAAAAAAfzM/cfrsM2h0RHA/s1600/ie-vpn-pro.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="32" src="http://1.bp.blogspot.com/-o4WeD6g3IuU/UnnFPSCSVnI/AAAAAAAAfzM/cfrsM2h0RHA/s640/ie-vpn-pro.exe.cropped.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-DfpULnimYqg/UnnFOpqm3VI/AAAAAAAAfy8/8Re_N5Ymnv0/s1600/ie-vpn-pro.padded.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://2.bp.blogspot.com/-DfpULnimYqg/UnnFOpqm3VI/AAAAAAAAfy8/8Re_N5Ymnv0/s640/ie-vpn-pro.padded.exe.cropped.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-YYE5hapMNNw/UnnFOXuzCLI/AAAAAAAAfy4/NKr3Q8wG6HQ/s1600/ie-vpn-pro.randpad.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://1.bp.blogspot.com/-YYE5hapMNNw/UnnFOXuzCLI/AAAAAAAAfy4/NKr3Q8wG6HQ/s640/ie-vpn-pro.randpad.exe.cropped.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-TiX0eV1o6yk/UnnFQTr41VI/AAAAAAAAfzY/-6qX8O1VUdc/s1600/ie-vpn-pro.signed_damaged.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://2.bp.blogspot.com/-TiX0eV1o6yk/UnnFQTr41VI/AAAAAAAAfzY/-6qX8O1VUdc/s640/ie-vpn-pro.signed_damaged.exe.cropped.png" width="640" /></a></div><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-vOAEraOR3qk/UnnFPQFvWMI/AAAAAAAAfzE/G2oGkEvqD_A/s1600/ie-vpn-pro.signed.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="65" src="http://4.bp.blogspot.com/-vOAEraOR3qk/UnnFPQFvWMI/AAAAAAAAfzE/G2oGkEvqD_A/s640/ie-vpn-pro.signed.exe.cropped.png" width="640" /></a><a href="http://3.bp.blogspot.com/-c0dOpSZjRrc/UnnFQwCxo-I/AAAAAAAAfzg/K1ohtajlpWY/s1600/ie-vpn-pro.signedrm.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="32" src="http://3.bp.blogspot.com/-c0dOpSZjRrc/UnnFQwCxo-I/AAAAAAAAfzg/K1ohtajlpWY/s640/ie-vpn-pro.signedrm.exe.cropped.png" width="640" /></a></div><br />Like all the other security suites, the original was flagged as a virus and removed. What happens with the transformed versions is rather more interesting. Firstly the padded version was still detected as a virus, while the randomly padded one wasn't (the random padding was longer than the non-random padding). Unique to IE the damaged signature transformation was reported as a 'corrupt or invalid' signature and treated differently to the random padding transformation. The self-signed Authenticode transformation was flagged as "not commonly downloaded" rather than a virus, and the the trusted certificate Authenticode transformation was flagged the same way but with a yellow bar rather than a red one.<br /><br />The diversity of messages here was surprising, clearly the signature of the file is being examined and being combined with some cloud based binary reputation system (Smartscreen filter?) before a determination is given to the user. It's worth noting that a non-malicious, unsigned, uncommon binary gave the same message as the signed (untrusted) executable and that a non-malicious, unsigned, common binary (putty.exe) gave no warning message. This means that the malicious, signed binary landed somewhere in between these two cases.<br /><br /><b>Part 6 - Conclusion</b><br /><div>First a caveat: with only a single malware sample and a small handful of security suites we can not come to any sweeping conclusions.</div><div><br /></div><div>However, it looks like Windows Defender / Internet Explorer as well as Sophos take into account Authenticode signatures when scanning executables. All tested security suites seem to have very fragile signature driven engines that were defeated by almost any change to the sample but these systems are backed up by heuristic systems that are at least partially powered by a cloud based binary reputation mechanism. Windows Defender and Sophos both differentiated between untrusted Authenticode signatures and trusted signatures and Windows Defender differentiated between Authenticode signatures, a corrupted Authenticode signature, and arbitrary appended data.</div><div><ul></ul></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com3tag:blogger.com,1999:blog-1404660204966454042.post-13393234897732122472013-11-05T19:00:00.002-08:002013-11-05T21:17:04.893-08:00A Change!<a href="http://www.riotgames.com/sites/all/themes/riot/media/images/logo.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://www.riotgames.com/sites/all/themes/riot/media/images/logo.jpg" /></a>Hey everyone, just dropping in to tell you that I'm moving from <a href="http://www.google.com/security">Google</a>&nbsp;to <a href="http://www.riotgames.com/">Riot Games</a>. I've loved my time at Google but I'm really excited to be able to work on security in the context of online games (also I'm a huge fan of&nbsp;<a href="http://www.leagueoflegends.com/">League of Legends</a>).&nbsp;I'll still be blogging (possibly more so than before) and as always the views on this blog represent my opinions and not that of my employer.<br /><div><br /></div>Stay tuned for a new blog post very soon now, I have some notes compiled that I just have to polish.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-64919749882581505722013-01-18T14:39:00.003-08:002013-01-18T14:39:32.772-08:00Hardcode 2013 Starts Today!<div class="separator" style="clear: both; text-align: center;"><a href="https://hardcode.googlecode.com/git/logo_2013.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="120" src="https://hardcode.googlecode.com/git/logo_2013.png" width="320" /></a></div><a href="https://code.google.com/p/hardcode/wiki/Hardcode2013ContestDescription" target="_blank">Hardcode 2013</a>,&nbsp;Google and Syscan's secure coding competition,&nbsp;has started! The contest information has been posted at <a href="https://code.google.com/p/hardcode/wiki/Hardcode2013ContestDescription">https://code.google.com/p/hardcode/</a><br /><br />From the description:<br /><blockquote class="tr_bq">Teams must develop a marketplace web application that allows people to organize bartering of academic goods or services in a school setting (e.g., selling used books, supplies, tutoring services). The Application should support a general marketplace where any Seller can post an Item they want to sell and any Buyer can express interest in or bid on an item. This Application does NOT include a payment transaction system; the Application connects potential Buyers with Sellers but does not perform actual payment transactions.</blockquote>If you're a student take a look!<br /><br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-37971473312207443712013-01-10T12:02:00.000-08:002013-01-10T12:02:52.031-08:00Hardcode: Google and Syscan's secure coding competitionGoogle and Syscan are running a secure coding competition with sizable cash prizes. Teams of students &nbsp;will build web applications of App Engine that will be judged on their features and overall security.<br /><br />Original post:&nbsp;<a href="http://googleonlinesecurity.blogspot.com/2013/01/calling-student-coders-hardcode-secure.html">http://googleonlinesecurity.blogspot.com/2013/01/calling-student-coders-hardcode-secure.html</a>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-32879911066851281312012-11-17T00:08:00.002-08:002012-11-17T00:08:22.802-08:00TextHole Source CodeA quick update.<div>&nbsp;The source code to&nbsp;<a href="http://memeover.arkem.org/2012/09/texthole.html" target="_blank">TextHole</a>&nbsp;is now available from <a href="https://github.com/arkem/texthole" target="_blank">my github account</a>.&nbsp;</div><div><div><br /></div></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-85357731181513311092012-11-05T10:16:00.001-08:002012-11-05T10:16:37.250-08:00Tavis Ormandy's (second) Sophail paperTavis has done it again with another paper about the failings of Sophos. This time with several interesting bugs and a working exploit.<br /><br />Paper:<br /><a href="https://lock.cmpxchg8b.com/sophailv2.pdf" target="_blank">Sophail: Applied attacks against Sophos Antivirus</a><br /><br />Full Disclosure Post (including link to exploit):<br /><a href="http://lists.grok.org.uk/pipermail/full-disclosure/2012-November/088813.html" target="_blank">[Full-disclosure] multiple critical vulnerabilities in sophos products</a><br /><br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-78793246312099561252012-09-15T15:44:00.003-07:002012-09-15T15:46:35.097-07:00TextHoleTo experiment with <a href="http://appengine.google.com/" target="_blank">Google Appengine</a>&nbsp;I've created a simple text repository application called <a href="http://texthole.arkem.org/" target="_blank">TextHole</a>.<br /><br />TextHole is a basic&nbsp;text repository with the following features:<br /><br /><ul><li>Anonymous uploads and read access</li><li>Optional Google OAuth2 authentication to allow you to delete or edit your uploads</li><li>A simple JSON interface makes it easy to post and download text from other sites</li></ul><div>To download text via JSON make a GET request to http://texthole.arkem.org/download/[mesage_id]<message_id></message_id></div><div>The reply will be a JSON dictionary with the following keys:</div><div><span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span></div><div><span style="font-family: Courier New, Courier, monospace;">body: the text body of the message</span></div><div><span style="font-family: Courier New, Courier, monospace;">editable: whether the requestor can modify the text</span></div><div><span style="font-family: 'Courier New', Courier, monospace;">creation: Creation time of the text</span></div><div><span style="font-family: Courier New, Courier, monospace;">expiry: unix timestamp of the expiry of the text</span></div><div><span style="font-family: 'Courier New', Courier, monospace;">message_id: the message id of this text</span></div><div><span style="font-family: Courier New, Courier, monospace;">status: True if the request succeeded</span></div><div><span style="font-family: Courier New, Courier, monospace;">error: If status is false more details here</span></div><div><span style="font-family: Courier New, Courier, monospace;"><br /></span></div><div><span style="font-family: Courier New, Courier, monospace;">Note: Only status and message_id fields are guaranteed</span></div><div><br /></div><div>To upload text via JSON make a POST request to http://texthole.arkem.org/upload providing a JSON dictionary via the data form field.&nbsp;</div><div><br /></div><div>Possible actions are:</div><div><span style="font-family: 'Courier New', Courier, monospace;">New message: The body key is required</span></div><div><div><span style="font-family: 'Courier New', Courier, monospace;">Delete: The delete key is required</span></div><div><span style="font-family: 'Courier New', Courier, monospace;">Edit message: The body and overwrite keys are required</span><br /><b style="font-family: 'Courier New', Courier, monospace;"><br /></b></div></div><div><div>Request dictionary keys:</div><div><div><span style="font-family: Courier New, Courier, monospace;">body: the text body of the new/modified message</span></div><div><span style="font-family: Courier New, Courier, monospace;">delete: the message id of the message to delete</span></div><div><span style="font-family: Courier New, Courier, monospace;">overwrite: the message id of the message to edit</span></div><div><span style="font-family: Courier New, Courier, monospace;">authenticated: if set attribute the new message to the user</span></div><div><span style="font-family: Courier New, Courier, monospace;">expiry: number of seconds (max 1yr) the text is to be valid for</span></div><div><span style="font-family: Courier New, Courier, monospace;"><br /></span></div><div><span style="font-family: Courier New, Courier, monospace;">Notes: One of body and delete is required. Overwrite and delete require a valid cookie to be sent with the request.</span></div><div><br /></div>Reply dictionary keys:&nbsp;</div><div><span style="font-family: 'Courier New', Courier, monospace;">message_id: the message id of the new/edited/deleted text</span></div><div><span style="font-family: Courier New, Courier, monospace;">status: True if the request succeeded</span></div><div><span style="font-family: Courier New, Courier, monospace;">error: If status is false more details here</span></div></div><div><span style="font-family: 'Courier New', Courier, monospace;">expiry: unix timestamp of the expiry of the text</span></div><div><span style="font-family: 'Courier New', Courier, monospace;">user: username of the owner of the text ("None" for anonymous)</span></div><div><br /></div><div>TextHole is missing the following features (maybe coming soon):</div><div><ul><li>An index of available texts</li><li>Text search</li><li>A javascript client library to make it even easier to integrate with TextHole</li><li>A way to authenticate via the JSON library</li></ul><div>Please play around with TextHole and send me any bugs or ideas that you find. Please remember that everything in TextHole is public, I can see it, and so can everyone else. Finally, please don't use TextHole for evil.</div></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-71696061509290627502012-09-07T08:49:00.000-07:002012-09-07T08:49:34.626-07:00Google acquires VirusTotal<div class="separator" style="clear: both; text-align: center;"><a href="https://www.virustotal.com/static/img/logo.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="50" src="https://www.virustotal.com/static/img/logo.png" width="320" /></a></div>VirusTotal, the online service that will scan uploaded files against dozens of AV engines has been acquired by Google.&nbsp;<a href="http://blog.virustotal.com/2012/09/an-update-from-virustotal.html" target="_blank">Here's the announcement.</a>&nbsp;I think this is great, I'm a big fan of VirusTotal and I am looking forward to what Google and VT can come up with together.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-63074951537873374582012-07-30T14:50:00.001-07:002012-07-30T14:50:13.476-07:00Owning Ubisoft<div class="separator" style="clear: both; text-align: center;"><a href="http://images4.wikia.nocookie.net/__cb20110831013736/assassinscreed/images/f/f6/UPLAY_logo_-_Small.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="100" src="http://images4.wikia.nocookie.net/__cb20110831013736/assassinscreed/images/f/f6/UPLAY_logo_-_Small.png" width="200" /></a></div>Tavis Ormandy is at it again, this time offhandedly revealing a drive-by code execution vulnerability in Ubisoft's Uplay platform. A malicious website could cause the Uplay browser plugin to execute arbitrary commands on the victim's computer. The attack takes advantage of a feature that allows a visited website to launch a Ubisoft game but does not check that the command that the website issues corresponds to a legitimate game. The issue has been patched in an emergency update from Ubisoft.<br /><br />Full details:&nbsp;<a href="http://seclists.org/fulldisclosure/2012/Jul/375">http://seclists.org/fulldisclosure/2012/Jul/375</a>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-21252144166175568012012-06-17T16:10:00.000-07:002012-06-17T16:14:17.738-07:00Mapping the relationship between YouTube videos<div class="separator" style="clear: both; text-align: center;"><br /></div>I've been playing John Robertson's YouTube choose your own adventure game <a href="http://www.youtube.com/watch?v=hvkjP6dqpfY">The Dark Room</a>&nbsp;and I've been having a great time. However, I need a little help navigating the room (you see, it's dark in there) and so I wrote a program to do a little cartography and create a map of the game.<br /><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://arkem.org/ytvidmap.svg" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" height="230" src="http://arkem.org/ytvidmap.svg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">An abbreviated map of <a href="http://www.youtube.com/watch?v=hvkjP6dqpfY">The Dark Room</a>&nbsp;(you can make a complete one with ytvidmap.py)</td></tr></tbody></table><div>The map shows the videos that comprise The Dark Room (abbreviated here for space and to limit the spoilers) with the size of each node proportional to the number of views the video has and the colour signifying the number of outbound links from the video. The map was generated by <a href="https://github.com/arkem/ytmap/blob/master/ytvidmap.py">ytvidmap.py</a> from my <a href="https://github.com/arkem/ytmap">ytmap</a> repository and is created by processing the YouTube annotations. Sadly, the annotations aren't available from the YouTube GData API so I process the annotations with regular expressions. The map provides a huge boon in navigating The Dark Room but does not make escaping trivial (it's like John anticipated this kind of analysis).</div><div><br /></div><div>After creating ytvidmap.py I realised that this approach could also be used to help me discover YouTube content by seeing who my favourite film makers and musicians linked to and in turn who they linked to. So I created <a href="https://github.com/arkem/ytmap/blob/master/ytusermap.py">ytusermap.py</a>&nbsp;and started by plotting the people in <a href="http://www.youtube.com/user/lindseystomp">Lindsey Stirling's</a> YouTube video social network and ended up with a giant mess of relationships that quickly got out of control. After adjusting my scripts to build in some limits I ended up with this diagram of her closest neighbors.</div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://arkem.org/ytusermap.svg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="186" src="http://arkem.org/ytusermap.svg" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;"><a href="http://www.youtube.com/user/lindseystomp">Lindsey Stirling's</a> YouTube collaboration social network&nbsp;</td></tr></tbody></table><div>While not the most useful&nbsp;analysis&nbsp;tool I've ever built I've been having fun with it and you should too!</div><div><br /></div><div>Check out <a href="https://github.com/arkem/ytmap">ytmap</a> on github!&nbsp;</div><div><br /></div><div>If you like spoilers here is a complete (as far as I know) version of <a href="http://arkem.org/ytvidmap_full.svg" target="_blank">The Dark Room map</a>&nbsp;(use your browser's zoom function to navigate it better).&nbsp;</div><div><br /></div><div>Finally, here is a large version of <a href="http://arkem.org/ls_big.svg" target="_blank">Lindsey Stirling's network</a> and a large version of <a href="http://arkem.org/gas_big.svg" target="_blank">GeekandSundry's network</a>&nbsp;(<a href="http://geekandsundry.com/" target="_blank">Felica Day and Will Wheaton's YouTube channel</a>).<br /><br />Edit: Viewing the images directly makes them clickable, so that they can take you directly to the YouTube user or video directly.</div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-29719517509659740902012-03-12T18:27:00.000-07:002012-03-12T18:27:40.228-07:00Identifying computers behind NAT with plotpcapFollowing on from my last post&nbsp;<a href="http://memeover.arkem.org/2012/02/identifying-computers-behind-nat-with.html">Identifying computers behind NAT with pyflag</a>&nbsp;I've made a stand alone script <a href="https://github.com/arkem/plotpcap">plotpcap</a>&nbsp;that can produce similar graphs without needing to install pyflag.<br /><br />The results aren't as pretty and you miss out on some of pyflag's analytical tools (such as filtering streams by user agents). On the other hand you do gain the ability to filter your output by tcpdump style filter strings and with a little bit of pcap preprocessing from tshark you can perform almost all the same comparisons.<br /><br />plotpcap requires the python modules dpkt, pcap (from pypcap) and matplotlib. I used the versions available from the Ubuntu 10.04 repository but other versions are probably good too.<br /><br />Here's some output generated from the same example data as the last post:<br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-GWk_XzYXK8M/T16eWHhXxXI/AAAAAAAABBE/Nah6VGDc5V4/s1600/ipid.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-GWk_XzYXK8M/T16eWHhXxXI/AAAAAAAABBE/Nah6VGDc5V4/s640/ipid.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number (note that without stream highlighting it gets a bit hard to read)</td></tr></tbody></table><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-u52a0hMw9Hs/T16ehuxhexI/AAAAAAAABBM/HEsRJ0SbaFY/s1600/ipid2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-u52a0hMw9Hs/T16ehuxhexI/AAAAAAAABBM/HEsRJ0SbaFY/s640/ipid2.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number after excluding packets with TCP timestamp options (ipid2)</td></tr></tbody></table><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-F_vEUHea5vU/T16ewLXw3nI/AAAAAAAABBU/H8XhUMzWrOo/s1600/tcptsval.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://3.bp.blogspot.com/-F_vEUHea5vU/T16ewLXw3nI/AAAAAAAABBU/H8XhUMzWrOo/s640/tcptsval.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">TCP Timestamps versus Packet Number</td></tr></tbody></table>If you wanted to do some of the tricks from the last post you can apply wireshark display filters to the pcap and then run it through plotpcap. For example:<br /><br /><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">tshark -r test.pcap -w test_chrome.pcap -R "http.user_agent contains Chrome"</span><br /><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">python plotpcap.py test_chrome.pcap number ipid</span><br /><span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span style="font-family: inherit;">Produces something like:</span><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-i5o0OPxjCp8/T16hwicTWSI/AAAAAAAABBc/J9u67yKD-DQ/s1600/ipid_chrome.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://3.bp.blogspot.com/-i5o0OPxjCp8/T16hwicTWSI/AAAAAAAABBc/J9u67yKD-DQ/s640/ipid_chrome.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number after matching the wireshark display filter "http.user_agent contains Chrome"</td></tr></tbody></table>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com4tag:blogger.com,1999:blog-1404660204966454042.post-13762456024497610462012-02-20T01:55:00.000-08:002012-03-12T18:29:03.329-07:00Identifying computers behind NAT with pyflagI've been a bit busy recently as I'm preparing to move across the world to the US to work at a <a href="http://www.google.com/">small Internet company</a> in the SF Bay Area. In the mean time though my current employer has been kind enough to let me contribute back some of the code we have written for the <a href="https://github.com/arkem/pyflag">pyflag</a>&nbsp;project (the link goes to my github page which has a fork of the project as the upstream site <a href="http://pyflag.net/">pyflag.net</a> is down right now). Update: An alternate version (without the feature described below) <a href="http://code.google.com/p/pyflag/">is available on google code</a><br /><br />The new features&nbsp;centre&nbsp;around identifying computers that are all lumped together behind a network address translation gateway (NAT). The idea is if you can identify the computers behind the NAT gateway you can attribute traffic to a specific system rather than only down to the network itself. The implementation is some visualisation tools in pyflag that allow you to plot certain packet headers fields against packet numbers or time.<br /><br />Here's an example:<br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-c1umPXBPCMU/T0H3YxWGOeI/AAAAAAAAA_0/ptp1Yg_WsQk/s1600/all+the+traffic.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="340" src="http://2.bp.blogspot.com/-c1umPXBPCMU/T0H3YxWGOeI/AAAAAAAAA_0/ptp1Yg_WsQk/s640/all+the+traffic.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID field plotted against PCAP packet number</td></tr></tbody></table>The plot takes the IP Identification field from the IP header and plots it sequentially against the PCAP packet number (pyflag also supports plotting against time). It looks like a big mess but you can see some lines and maybe some patterns in there. The IPID field is used to associate fragmented packets together for reassembly and it is generally left untouched by NAT gateways. Usefully different networking stacks have different strategies for picking IPID values.<br /><br />In my anecdotal (non-scientifically determined) experience:<br /><br /><ul><li>&nbsp;Windows machines start at 0 when the computer is booted and increment for each packet sent up until 2^16 and then start again. In some cases it seems to wrap at 2^15 which to me suggests a signed integer problem but I haven't conclusively figured out on what versions it happens on. Additionally, I've read (but not seen) that some versions of Windows send the field in host order rather than network byte order.</li><li>Linux machines pick a random number for the start of the connection and then increment the value for each subsequent packet of the connection. I've heard (but again not seen) that packets with the Don't Fragment bit set get their IPID set to 0 on Linux.</li><li>BSD machines (including Mac OS X) pick a random number for every packet.</li></ul><div>So looking back at our example we can see a haze of small lines and also a couple of longer lines which suggests that we might be looking at one or more Linux boxes along with one or more Windows boxes. To test this theory I looked for any user-agent strings in web traffic and found the following:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-U3NiM7ReItE/T0H6S2ishvI/AAAAAAAAA_8/swUaLXgf9vA/s1600/user+agents+cropped.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="564" src="http://4.bp.blogspot.com/-U3NiM7ReItE/T0H6S2ishvI/AAAAAAAAA_8/swUaLXgf9vA/s640/user+agents+cropped.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">User-Agent strings present in the sample PCAP file</td></tr></tbody></table><div>Based on those user agent strings it looks like there is at least one Ubuntu system and one Windows system. Also of note is the presence of Java user agent strings as well as Transmission (the Ubuntu Bittorrent client).</div><div><br /></div><div>If we revisit our previous IPID plot and tell pyflag to colour all the Chrome/Windows user agent string related streams blue we get the following:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-HFBIeYT_VWc/T0ID9P0rItI/AAAAAAAABAE/FbnVUy9CqXw/s1600/windows+chrome+plot+2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://3.bp.blogspot.com/-HFBIeYT_VWc/T0ID9P0rItI/AAAAAAAABAE/FbnVUy9CqXw/s640/windows+chrome+plot+2.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IP ID versus PCAP number with Chrome on Windows streams highlighted</td></tr></tbody></table><div>From this it becomes clear that there are two distinct lines of IPID growth which implies that behind this NAT gateway are two Windows systems, one which was active for longer and even sent enough packets that the IPID value wrapped. Knowing the shape of these lines means that you can associate other traffic (perhaps traffic with no distinguishing application layer features such as encrypted streams) to a specific computer and any metadata gleamed from other application protocols (like HTTP). &nbsp;</div><div><br /></div><div>To make this even clearer there's another header field to consider, this time in the TCP header. There is an optional header in TCP called the timestamp value (defined by RFC1323) which is used to measure packet round trip times. By default Windows systems omit this value while most other systems include it (I've read that Windows can be configured to send timestamps and that in some cases will use timestamps if the client connecting to it uses timestamps). This means that if we exclude packets that have a TCP timestamp we should be left with all Windows traffic (assuming we exclude non-TCP traffic as well).</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-olCh29qLt9k/T0IGGUYnWqI/AAAAAAAABAM/sl6DlY_9r78/s1600/windows+chrome+plot+3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-olCh29qLt9k/T0IGGUYnWqI/AAAAAAAABAM/sl6DlY_9r78/s640/windows+chrome+plot+3.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID versus PCAP number for Chrome user-agents, minus packets that have a TCP timestamp</td></tr></tbody></table><div>After excluding packets with the TCP timestamp option set most of the background packets have been excluded. The remaining packets that don't fall on the lines are likely parser failures or packets generated by a Linux box that do not have a timestamp value for one reason or another (more investigation is required).</div><div><br /></div><div>So we're convinced that there are two Windows system on the network and some yet to be determined number of Linux systems, if we change our filter to highlight Firefox on Linux and then plot IPID we get something that looks like this:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-idWEbYDnp4k/T0IH_EUlkWI/AAAAAAAABAU/QxuifraQvcI/s1600/linux+web+traffic.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://4.bp.blogspot.com/-idWEbYDnp4k/T0IH_EUlkWI/AAAAAAAABAU/QxuifraQvcI/s640/linux+web+traffic.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">IPID versus PCAP number for Firefox sessions on Linux</td></tr></tbody></table><div>&nbsp;The things to note here is that the IPID values change dramatically between connections, also that in general HTTP traffic seems to be in the minority of the non-Windows traffic and finally that we're no closer to determining how many Linux systems are present. However, if we consider the TCP timestamp field for a moment we learn that it's generally determined as:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-_8geOqFETcI/T0IJKZgSSkI/AAAAAAAABAc/geyVTGM5YcI/s1600/timestamps.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="54" src="http://3.bp.blogspot.com/-_8geOqFETcI/T0IJKZgSSkI/AAAAAAAABAc/geyVTGM5YcI/s320/timestamps.png" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">From:&nbsp;<a href="http://www.slideshare.net/gwicherski/identifying-hosts-with-natfilterd">Identifying hosts with natfilterd</a></td></tr></tbody></table><div>The interesting part in this case is that wallclock - boottime should be unique among the hosts that use the TCP timestamp option and it should increment in a predictable fashion. So if we graph the TCP timestamp value of packets versus their PCAP number we get:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-GBK7r6AV6PU/T0IJ2YJPNzI/AAAAAAAABAk/DGWnjFPbOAs/s1600/linux+web+traffic+tsval.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-GBK7r6AV6PU/T0IJ2YJPNzI/AAAAAAAABAk/DGWnjFPbOAs/s640/linux+web+traffic+tsval.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">TCP Timestamp value versus PCAP packet number (Firefox/Linux traffic highlighted)</td></tr></tbody></table><div>Again we can see that the Firefox traffic accounts for only a minority of packets and we also see that there're two distinct lines for the first half on the plot. These two lines suggests that there are two Linux systems and the line fragment at the end probably represents a reboot (and not wrapping because the timestamp values are 32 bit numbers and the values we see are around 2^18 at their highest) of one of the systems or the appearance of a new one.</div><div><br /></div><div>So at this point I'm convinced that there are two Linux systems and two Windows system and that most of the Windows packets are HTTP traffic (using Chrome) and that while there is HTTP traffic it accounts for only a small amount of the Linux related packets. For the remainder of the Linux traffic I'd guess that at least one of the systems is transferring files using BitTorrent based on the Transmission user-agent that was present before. Maybe if we plot the traffic with the Transmission user-agent we'll be able to tell which computers were running BitTorrent:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"></div><br /><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-ZtUKPUdU3h8/T0IQJyNpc8I/AAAAAAAABA0/iXWKa0Nk7vo/s1600/transmission.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://1.bp.blogspot.com/-ZtUKPUdU3h8/T0IQJyNpc8I/AAAAAAAABA0/iXWKa0Nk7vo/s640/transmission.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">TCP Timestamp versus PCAP Packet Number for the user-agent "Transmission"</td></tr></tbody></table><div>&nbsp;At first this looks good, the line with the lower timestamp values is associated with Transmission and the higher one is not. Unfortunately this plot is ambiguous because the third line section is also associated with Transmission traffic and that line could easily belong to the top line section (after a reboot). If instead we ask pyflag to generate a table with only traffic that is not to or from ports 80 or 53 (to eliminate HTTP and DNS) we're left with a lot of connections between high ports transferring lots of &nbsp;encrypted (looking) data to our NAT gateway address which fits the hypothesis of BitTorrent traffic. When we plot the timestamp values again and highlight any packet from our Not-HTTP/Not-DNS table we get the following:</div><div><br /></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody><tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-lxQJnMgc6KA/T0ITcUpv_aI/AAAAAAAABA8/Wy0ShSYreCM/s1600/non-http+non-dns.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-lxQJnMgc6KA/T0ITcUpv_aI/AAAAAAAABA8/Wy0ShSYreCM/s640/non-http+non-dns.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">TCP Timestamp versus PCAP number with non-HTTP/non-DNS traffic highlighted</td></tr></tbody></table><div>At this point I'm reasonably confident that both the observed Linux hosts are downloading files over BitTorrent once I combine this plot with some analysis of the ports / stream sizes seen while I'm equally convinced that the Windows systems are not using BitTorrent or at least that there isn't a significant level of BitTorrent traffic observed during this packet capture.</div><div>The above little demo is contrived but I have found that this kind of analysis can be really useful in characterising the use of a network. This example was constructed from 5 virtual machines, 2 running Windows XP, 2 running Ubuntu 10.04 and a NAT gateway running Ubuntu 10.04 and using iptables/netfilter to do the NATing. Also, just in case you were wondering the Windows machines were watching youtube (in particular <a href="http://www.youtube.com/watch?v=wZZ7oFKsKzY">nyan cat</a> and <a href="http://www.youtube.com/watch?v=WG60-0tp5sU">techno viking</a>) while the Ubuntu systems were each using BitTorrent to download ubuntu images (12.04 alpha for different architectures).&nbsp;</div><div><br /></div><div><br /></div><div>Future Work</div><div><ul><li>Spring cleaning of the pyflag source (it's a little annoying to build and use right now)</li><li>More options on what to graph (maybe a system for generically plotting table information)</li><li>Ability to choose what to highlight based off the reverse side of a stream</li><li><strike>Implementing a minimal version of this visualisation outside of pyflag</strike>&nbsp;Done!&nbsp;<a href="http://memeover.arkem.org/2012/03/identifying-computers-behind-nat-with.html">Identifying computers behind NAT with plotpcap</a></li></ul><div><br /></div><div>Related Work and Further Reading</div></div><div><ul><li><a href="http://nmap.org/book/osdetect-methods.html">nmap book - os detection</a></li><li><a href="http://lcamtuf.coredump.cx/p0f3/">lcamtuf's p0f3 fingerprinting tool</a></li><li><a href="http://www.slideshare.net/gwicherski/identifying-hosts-with-natfilterd">Georg Wicherski's Identifying hosts with natfilterd</a></li><li><a href="https://www.cs.columbia.edu/~smb/papers/fnat.pdf">Steven M. Bellovin's A Technique for Counting NATed Hosts</a></li><li><a href="http://www.phrack.org/issues.html?id=3&amp;issue=63">Elie aka Lupin TCP Timestamp to count hosts behind NAT (Phrack 63)</a></li></ul><div><br /></div><div>Update:</div></div><div>Now that I've got the links handy I thought I'd also point at Michael Cohen's work. Michael is one of the authors of pyflag (project lead is probably a better description), and it's his ideas and that lead to the implementation of IP ID processing in pyflag.</div><div><ul><li><a href="https://docs.google.com/viewer?a=v&amp;pid=explorer&amp;chrome=true&amp;srcid=0B9hc84IflFGbODRjMzc4ZjgtNWJiNS00NWRlLWJhYjQtZTk3Mjg1ODc0ODA3&amp;pli=1#">Source attribution for network address translated forensic captures</a></li><li><a href="https://docs.google.com/viewer?a=v&amp;pid=explorer&amp;chrome=true&amp;srcid=0B9hc84IflFGbZTkyODRjYzUtNzNiYi00MzNhLWI3OTEtM2M4ZWRkMjQzZDk4&amp;pli=1#">Network forensics - A Practical Introduction</a></li></ul></div><div><br /></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-53928998361091931452012-01-02T18:09:00.000-08:002012-01-02T18:09:50.213-08:00Yet Another First Ascension Post<div class="separator" style="clear: both; text-align: left;">I was going through the pages of an old defunct blog of mine and I saw this image and thought that I would repost it for old times sake. This is one of my proudest computer gaming moments of all time (from October 2009).</div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="http://arkem.org/nethack/score.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="534" src="http://arkem.org/nethack/score.png" width="640" /></a></div><br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com2tag:blogger.com,1999:blog-1404660204966454042.post-74540477827037023482011-09-06T05:44:00.001-07:002011-09-06T05:46:17.413-07:00Something you should know about talloc<a href="http://talloc.samba.org/talloc/doc/html/index.html">Talloc</a>&nbsp;is an excellent memory management system for C that provides&nbsp;hierarchical memory pools with other cool tricks like&nbsp;destructors. It's written by <a href="http://en.wikipedia.org/wiki/Andrew_Tridgell">Tridge</a>&nbsp;for&nbsp;Samba and I really like it. If you are writing a complex system in C you could do worse than to replace your calls to malloc with calls to talloc.<br /><br />So that's talloc, but the thing you really should know about talloc is right there at the bottom of the project page. In particular:<br /><span style="font-family: 'Courier New', Courier, monospace;"></span><br /><div><span style="font-family: 'Courier New', Courier, monospace;"><span style="font-family: 'Courier New', Courier, monospace;"><br /></span></span></div><span style="font-family: 'Courier New', Courier, monospace;">when using talloc_enable_leak_report(), giving directly NULL as a parent context implicitly refers to a hidden "null context" global variable, so this should not be used in a multi-threaded environment without proper synchronization.</span><br /><div><span style="font-family: 'Courier New', Courier, monospace;"><br /></span></div><div><span style="font-family: inherit;">I've spent many days recently hunting down a bug, the bug would have been much easier to find if I had read the above line. Suddenly I was sharing contexts all over the place and <i>very very rarely</i>&nbsp;there'd be a synchronization problem that would lead to a null pointer deref.&nbsp;</span></div><div><span style="font-family: inherit;"><br /></span></div><div><span style="font-family: inherit;">By the way talloc_enable_leak_report() is an excellent feature of talloc. Excellent.&nbsp;</span></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-59350502038529436272011-08-08T04:43:00.000-07:002013-11-07T18:11:39.807-08:00Authenticode and Antivirus Detection part 2<b>Update Nov 2013: </b>Another follow up post: <a href="http://memeover.arkem.org/2013/11/authenticode-and-antivirus-detection.html">Authenticode and Antivirus Detection Revisited</a><br /><br />After Shane's comments on the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection.html">Authenticode and Antivirus Detection</a>&nbsp;post I thought I'd run some more tests. I wanted to try and figure out how much of the observed detection difference were because some extra bytes had been added and how much was due to special handling of signed binaries.<br /><br />I found an archive of malware online and created four sets of samples. Set one was the malware without any changes, set two was after the binaries had been signed with the TEST1 certificate, set three was signed with a TEST2 certificate that was similar to TEST1 but was only valid from 1975 - 2009 and set four had a random blob of 32 bytes appened to the end. Using the VirusTotal API and <a href="http://www.bryceboe.com/2010/09/01/submitting-binaries-to-virustotal/">Bryce Boe's python script</a>&nbsp;I ran each of the sets against the VirusTotal antivirus suite.<br /><br />The resulting statistics are <a href="http://arkem.org/amag_results_100_sorted.txt">here</a>, showing the number of AV positives, the format is:<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">&nbsp;"HASH [SET1, SET2, SET3, SET4] [SET1 - SET2, SET1 - SET4]"</span><br /><br />And here are the first 10 entries (ordered by decreasing "SET1 - SET2" value):<br /><br /><pre style="white-space: pre-wrap; word-wrap: break-word;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">DB1D5E...34573 ['28', '10', '16', '22'] ['18', '6']<br />FDFB86...1FE0C ['34', '18', '18', '27'] ['16', '7']<br />6D48A7...F4880 ['36', '20', '22', '33'] ['16', '3']<br />CA9C3E...ED072 ['31', '16', '16', '23'] ['15', '8']<br />8798FA...8755B ['35', '20', '20', '32'] ['15', '3']<br />1011ED...0DB18 ['35', '20', '20', '33'] ['15', '2']<br />DA01D0...C899D ['31', '17', '16', '28'] ['14', '3']<br />CC3B7D...228D1 ['37', '23', '23', '34'] ['14', '3']<br />CADD90...CE9C4 ['35', '21', '21', '31'] ['14', '4']<br />B6BBE8...8CD10 ['32', '18', '18', '29'] ['14', '3']</span></pre><br /><span class="Apple-style-span" style="font-family: inherit;">General observations:</span><br /><ul><li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">Adding either an authenticode signature or random data would defeat several engines</span></span></li><li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">Very rarely would the signing certificate's validity influence the score</span></span></li><li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">For some reason adding the random data occasionally resulted in more signatures being hit and considering that the same data was added to each sample I'm not sure what happened there.</span></span></li><li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">This test primarily tests the AV signature engines and not their runtime or heuristic scanners</span></span></li><li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">The VirusTotal API limit of 20 requests each 5 minutes sounds like a lot until you run tests like this.</span></span></li></ul><div>Really what I've learnt from this is that AV signatures are even more fragile than I realised. To get a proper look at how AV treats authenticode signed binaries I think I'd need to evaluate all of each AV's modules and not just the signature engine.</div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-22949405601975732852011-08-06T04:35:00.004-07:002011-08-06T15:18:00.999-07:00Tavis Ormandy's Sophail paperOn the topic of antivirus, Tavis Ormandy has recently released a paper looking into the internals of Sophos. It's quite scathing and very interesting, check it out:&nbsp;<a href="http://lock.cmpxchg8b.com/Sophail.pdf">Sophail: A Critical Analysis of Sophos Antivirus</a>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-76150785027671327422011-08-06T04:05:00.000-07:002013-11-07T18:12:30.298-08:00Authenticode and Antivirus Detection<b>Update Nov 2013: </b>Follow up posts:&nbsp;<a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection_08.html">Authenticode and Antivirus part 2</a>, <a href="http://memeover.arkem.org/2013/11/authenticode-and-antivirus-detection.html">Authenticode and Antivirus Detection Revisited</a><br /><br />It turns out that many antivirus engines white list authenticode signed binaries regardless of the trustworthiness of the signature. Here's an experiment that I performed, feel free to play along at home (remember to be careful when working with malware).<br /><br /><b>Step 1</b>: Find some malware<br />This was actually the most time consuming step, a lot of places talk about malware and offer large archives of malware samples to download. Even so, it took me a good 15 minutes to find a malicious windows executable that I could download from a site without a password, registration or other&nbsp;nonsense. In the end I found a site that lists live drive by download sites and I grabbed an EXE before the particular malware host went down. Sadly I can't find the link to the index site I was using, I'm sure a little bit of Googling will allow you to retrace my steps.<br /><br />I ended up with freedom.exe md5sum: ba87b562c829b7095bfb9e0db7a39890<br /><br /><b>Step 2</b>: Confirm that it is detected by Antivirus<br />For this to work you need to know that your malware sample is detected by antivirus engines so I recommend submitting it to&nbsp;<a href="http://virustotal.com/">VirusTotal</a>&nbsp;or similar service. Alternative if you have the resources run it against your local battery of antivirus installs.<br /><br />Freedom.exe was detected under a variety of names, Microsoft Security Essentials calls it&nbsp;Trojan:Win32/Danginex. The results were 36/43 (83.7%) considered Freedom.exe malware.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-1ADDJ_AxO-o/Tj0ZMY9XDwI/AAAAAAAAA90/L4oCXUMmU-s/s1600/freedom.exe-vt-2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="131" src="http://2.bp.blogspot.com/-1ADDJ_AxO-o/Tj0ZMY9XDwI/AAAAAAAAA90/L4oCXUMmU-s/s640/freedom.exe-vt-2.png" width="640" /></a></div><br /><b>Step 3</b>: Generate a code signing certificate<br />I don't have a proper code signing certificate handy so I thought I'd generate a self-signed certificate for the test. I used makecert.exe and pvk2pfx.exe from the Windows SDK 7.1 and the following commands:<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">makecert -r -pe -$ individual -n CN=TEST1 -sv test1.pvk test1.cer</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">pvk2pfx -pvk test1.pvk -spc test1.cer -pfx test1.pfx</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span class="Apple-style-span" style="font-family: inherit;"><b>Step 4: </b>Sign the malware sample</span><br /><span class="Apple-style-span" style="font-family: inherit;">Copy the sample to a new filename and then use signtool.exe to add the authenticode signature saying that TEST1 is responsible for this file.</span><br /><span class="Apple-style-span" style="font-family: inherit;"><br /></span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">signtool sign /f test1.pfx freedo-signed-test1.exe</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/-2YCL_WG3WZU/Tj0dGVXhyMI/AAAAAAAAA94/NqIrnrwlLLc/s1600/freedo-signed-test1.exe-signed.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="378" src="http://1.bp.blogspot.com/-2YCL_WG3WZU/Tj0dGVXhyMI/AAAAAAAAA94/NqIrnrwlLLc/s400/freedo-signed-test1.exe-signed.png" width="400" /></a></div><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span class="Apple-style-span" style="font-family: inherit;"><b>Step 5: </b>See what AV thinks of this new file</span><br /><span class="Apple-style-span" style="font-family: inherit;">Submit your new file to&nbsp;</span><a href="http://virustotal.com/">VirusTotal</a>&nbsp;and see what happens. In the case of Freedom.exe the detection rate fell from 83.7% to 27.9% (12/43). Most of the big names in the AV community (with a couple of notable exceptions) were quite happy to ignore Freedom.exe once it had been signed.<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="http://4.bp.blogspot.com/-_YWVVatS5qs/Tj0dyM8qJEI/AAAAAAAAA98/dULuyz5YJrc/s1600/freedo-signed-test1.exe-vt.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="130" src="http://4.bp.blogspot.com/-_YWVVatS5qs/Tj0dyM8qJEI/AAAAAAAAA98/dULuyz5YJrc/s640/freedo-signed-test1.exe-vt.png" width="640" /></a></div><br />The Antivirus engines that changed their minds about freedom.exe are:<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">AhnLab-V3,&nbsp;AVG,&nbsp;BitDefender,&nbsp;CAT-QuickHeal,&nbsp;Comodo,&nbsp;Emsisoft,</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">F-Secure,&nbsp;Fortinet,&nbsp;Ikarus,&nbsp;K7AntiVirus,&nbsp;McAfee,</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">McAfree-GW-Edition,&nbsp;Microsoft,&nbsp;Norman,&nbsp;nProtect,&nbsp;PCTools,&nbsp;Rising,</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Sophos,&nbsp;Symantec,&nbsp;TheHacker,&nbsp;TrendMicro,&nbsp;TrendMicro-HouseCall,</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">VIPRE,&nbsp;ViRobot</span><br /><br />Notably Kaspersky flagged both the original and modified samples as&nbsp;Trojan-Clicker.Win32.Agent.shx and ClamAV among 7 others did not flag either sample.<br /><br /><b>Conclusion: </b>What have we learnt?<br />Signed executables are more likely to be considered benign by antivirus engines. Signed executables are probably excluded by policy for performance reasons but it is possible (but unlikely) that instead that the addition of the Authenticode block at the end of the file is disrupting the signatures used by the engines. I hope that in the future that if vendors are going to exclude signed binaries that they at least check to see if the certificate used to sign the binary is trusted.<br /><div><br /></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com2tag:blogger.com,1999:blog-1404660204966454042.post-16206828933063793792011-05-26T02:44:00.000-07:002011-05-26T02:44:43.358-07:00GitHub additions!<div class="separator" style="clear: both; text-align: center;"></div><div class="separator" style="clear: both; text-align: center;"><a href="https://secure.gravatar.com/avatar/6c15f1ddc77d6ba95b6cf59c91f5a1a1?s=140&amp;d=https://d3nwyuy0nl342s.cloudfront.net%2Fimages%2Fgravatars%2Fgravatar-140.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="50" src="https://secure.gravatar.com/avatar/6c15f1ddc77d6ba95b6cf59c91f5a1a1?s=140&amp;d=https://d3nwyuy0nl342s.cloudfront.net%2Fimages%2Fgravatars%2Fgravatar-140.png" width="50" /></a></div><span class="Apple-style-span" style="font-family: inherit;">I've ported some of my old projects over to git and uploaded them to <a href="https://github.com/arkem">github</a>.</span><br /><span class="Apple-style-span" style="font-family: inherit;">A much better solution than hosting raw source files on my web server!</span><br /><br /><br />The projects that have been ported:<br /><span class="Apple-style-span" style="font-family: inherit;"><a href="https://github.com/arkem/talklikewarren">talklikewarren </a>-&nbsp;<span class="Apple-style-span" style="line-height: 16px;">A twitter bot that posts things that sound like Warren Ellis.</span></span><br /><span class="Apple-style-span" style="font-family: inherit; line-height: 16px;"><a href="https://github.com/arkem/middleman">fakemiddleman </a>- A twitter bot that posts things that sound like The Middleman.</span><br /><span class="Apple-style-span" style="font-family: inherit; line-height: 16px;"><a href="https://github.com/arkem/hottest100">hottest100 </a>- A python script that created a live music video channel out of The Triple J Hottest 100.</span><br /><span class="Apple-style-span" style="font-family: inherit;"><span class="Apple-style-span" style="line-height: 16px;"><a href="https://github.com/arkem/top1m">top1m </a>-&nbsp;</span><span class="Apple-style-span" style="line-height: 16px;">A squid redirector that prevents clients from visiting sites outside of the Alexa top 1 million.</span></span><br /><span class="Apple-style-span" style="font-family: inherit; line-height: 16px;"><a href="https://github.com/arkem/twitbot">twitbot&nbsp;</a>- An example of how one might use twitter as a channel for command and control of malware.</span><br /><span class="Apple-style-span" style="line-height: 16px;"><span class="Apple-style-span" style="font-family: inherit;"><br /></span></span><br /><div class="separator" style="clear: both; text-align: center;"></div><span class="Apple-style-span" style="line-height: 16px;"><span class="Apple-style-span" style="font-family: inherit;"><br /></span></span>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-38500730472514857622011-05-14T19:18:00.000-07:002011-05-14T19:18:32.620-07:00py360 - A speed increase (mostly)I've updated py360 (from the patch note):<br /><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Changed the Partition class from preprocessing the entire partition during its constructor, instead it now will resolve files and directories on demand and store the results for later. Basically trading precomputation for memoization. gamertags.py runs about 90x faster, report360.py runs about the same (since it touches every file) and mounting is about 100x faster. These improvements are at the cost of all first time reads being slightly slower but no wasted preprocessing is done.</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"><br /></span><br /><span class="Apple-style-span" style="font-family: inherit;">What isn't mentioned is that this will also make it much more likely that corrupted partitions will mount (which was why I started looking at this change at all). Also, gamertags.py and report360.py have been changed to be compatible with the new changes. The main difference is that using partition.allfiles does not necessarily return all files but rather all the processed files, use partition.walk() to get all files. The old behaviour is still available by passing in precache=True to the Partition constructor.&nbsp;</span><br /><br />This is a fairly experimental patch so let me know if it doesn't work for you. Next on the dev list remains better output from report360.py (and STFS / XDBF).<br /><br />I also hope to write some posts unrelated to py360 soon too!Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com2tag:blogger.com,1999:blog-1404660204966454042.post-8044090169967347552011-05-10T00:43:00.000-07:002011-05-10T00:43:10.162-07:00py360 - Update!Thanks to the feedback and test data provided by some excellent people (Thanks Juri, Matt and DC) I've managed to fix several bugs in <a href="https://github.com/arkem/py360">py360</a> and it should now be a smoother experience.<br /><br />The biggest fix is in the STFS parser which would naively assume that all filelisting blocks were contiguous, this fix means that STFS files with large numbers of files inside them will now work (e.g. Profiles that see a lot of use). Now report360.py should run cleanly on (hopefully) all images and the only errors you will see will be from trying to parse deleted files (they are recognisable by the ~ that precedes their name).<br /><br />Next up on the dev list is to investigate the unicode parsing problems that occasionally appear and cause extraneous bytes (usually nulls) to appear in the output. Worst case I plan on changing report360.py to remove the null bytes, best case I find the underlying cause and sort that out.<br /><br />If you would like to help with py360 I'd like to hear from you. I'm really interested in hearing about your experiences with py360 and especially tell me of any errors of inconsistencies that you encounter. If you're a programmer and really keen feel free to contribute code —especially example programs and bugfixes— I won't turn you away!Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-7344734258445662042011-05-07T01:02:00.002-07:002011-06-05T04:40:58.105-07:00py360 - An example to print out gamertagsEDIT: The code below is obsolete check <a href="https://github.com/arkem/py360">github</a>&nbsp;for an updated version.<br /><br />To help people get started with <a href="https://github.com/arkem/py360">py360</a>&nbsp;I written a simple example (simple at least compared to report360.py). gamertags.py takes an Xbox 360 disk image and prints out the gamertags of all the profiles present.<br /><br />It produces output like:<br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Gamertag: Han Solo, Type: Gold (Paid)</span><br /><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Gamertag: Chewbacca, Type: Silver (Free)</span><br /><br />The code is simple enough that I've included it below, it's also now in the git repository.<br /><br /><pre style="background: #000000; color: #d1d1d1;"><span style="color: #9999a9;"># An example of using py360 to extract all the GamerTags on a drive.</span><br /><br /><span style="color: #9999a9;"># Where does this data live?</span><br /><span style="color: #9999a9;"># GamerTags are inside the Account block of a user STFS container</span><br /><span style="color: #9999a9;"># located on the XTAF partition. </span><br /><br /><span style="color: #9999a9;"># How do you find user STFS containers?</span><br /><span style="color: #9999a9;"># Gamer profiles are located in the /Content directory in subdirectories</span><br /><span style="color: #9999a9;"># named with 16 hex characters starting with an 'E' such as</span><br /><span style="color: #9999a9;"># E00012DD5A4FAEE5. The STFS container is located in the </span><br /><span style="color: #9999a9;"># FFFE07D1/00010000 subdirectory and is named the same as the</span><br /><span style="color: #9999a9;"># profile directory.</span><br /><span style="color: #9999a9;"># Example: /Content/E00012DD5A4FAEE5/FFFE07D1/00010000/E00012DD5A4FAEE5 </span><br /><br /><span style="color: #e66170; font-weight: bold;">from</span> py360 <span style="color: #e66170; font-weight: bold;">import</span> partition<span style="color: #d2cd86;">,</span> stfs<span style="color: #d2cd86;">,</span> account<br /><span style="color: #e66170; font-weight: bold;">import</span> sys<br /><br /><span style="color: #9999a9;"># First, open the xbox 360 image</span><br />part <span style="color: #d2cd86;">=</span> partition<span style="color: #d2cd86;">.</span>Partition<span style="color: #d2cd86;">(</span>sys<span style="color: #d2cd86;">.</span>argv<span style="color: #d2cd86;">[</span><span style="color: #008c00;">1</span><span style="color: #d2cd86;">]</span><span style="color: #d2cd86;">)</span><br /><br /><span style="color: #9999a9;"># Second, find profile STFS containers</span><br /><span style="color: #e66170; font-weight: bold;">for</span> directory <span style="color: #e66170; font-weight: bold;">in</span> part<span style="color: #d2cd86;">.</span>allfiles<span style="color: #d2cd86;">[</span><span style="color: #00c4c4;">'/Content'</span><span style="color: #d2cd86;">]</span><span style="color: #d2cd86;">.</span>files<span style="color: #d2cd86;">:</span><br /> <span style="color: #e66170; font-weight: bold;">if</span> <span style="color: #e34adc;">len</span><span style="color: #d2cd86;">(</span>directory<span style="color: #d2cd86;">)</span> <span style="color: #d2cd86;">=</span><span style="color: #d2cd86;">=</span> <span style="color: #008c00;">16</span> <span style="color: #e66170; font-weight: bold;">and</span> directory<span style="color: #d2cd86;">[</span><span style="color: #008c00;">0</span><span style="color: #d2cd86;">]</span> <span style="color: #d2cd86;">=</span><span style="color: #d2cd86;">=</span> <span style="color: #00c4c4;">'E'</span><span style="color: #d2cd86;">:</span><br /> <span style="color: #9999a9;"># Open each STFS container and look for the Account block</span><br /> <br /> <span style="color: #9999a9;"># The STFS class can take either an actual file or a file-like object,</span><br /> <span style="color: #9999a9;"># we're using an file-like object to avoid having to use a temp file.</span><br /> path <span style="color: #d2cd86;">=</span> <span style="color: #00c4c4;">'/Content/%s/FFFE07D1/00010000/%s'</span> <span style="color: #d2cd86;">%</span> <span style="color: #d2cd86;">(</span>directory<span style="color: #d2cd86;">,</span> directory<span style="color: #d2cd86;">)</span><br /><br /> <span style="color: #9999a9;"># This test is to exclude deleted profiles and defunct directories</span><br /> <span style="color: #e66170; font-weight: bold;">if</span> path <span style="color: #e66170; font-weight: bold;">in</span> part<span style="color: #d2cd86;">.</span>allfiles<span style="color: #d2cd86;">:</span><br /> profile <span style="color: #d2cd86;">=</span> stfs<span style="color: #d2cd86;">.</span>STFS<span style="color: #d2cd86;">(</span>filename <span style="color: #d2cd86;">=</span> <span style="color: #e34adc;">None</span><span style="color: #d2cd86;">,</span> fd <span style="color: #d2cd86;">=</span> part<span style="color: #d2cd86;">.</span>open_fd<span style="color: #d2cd86;">(</span>path<span style="color: #d2cd86;">)</span><span style="color: #d2cd86;">)</span><br /><br /> <span style="color: #9999a9;"># The account block is always at /Account in the STFS archive</span><br /> <span style="color: #9999a9;"># we'll read it in, decode it and then print out the gamertag</span><br /> acc <span style="color: #d2cd86;">=</span> account<span style="color: #d2cd86;">.</span>Account<span style="color: #d2cd86;">(</span>profile<span style="color: #d2cd86;">.</span>read_file<span style="color: #d2cd86;">(</span>profile<span style="color: #d2cd86;">.</span>allfiles<span style="color: #d2cd86;">[</span><span style="color: #00c4c4;">'/Account'</span><span style="color: #d2cd86;">]</span><span style="color: #d2cd86;">)</span><span style="color: #d2cd86;">)</span><br /> <span style="color: #e66170; font-weight: bold;">print</span> <span style="color: #00c4c4;">"Gamertag: %s, Type: %s"</span> <span style="color: #d2cd86;">%</span> <span style="color: #d2cd86;">(</span>acc<span style="color: #d2cd86;">.</span>get_gamertag<span style="color: #d2cd86;">(</span><span style="color: #d2cd86;">)</span><span style="color: #d2cd86;">,</span> acc<span style="color: #d2cd86;">.</span>live_type<span style="color: #d2cd86;">)</span><br /></pre>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-15360340327345410032011-04-16T23:49:00.000-07:002011-04-16T23:49:16.494-07:00py360 - Xbox 360 File System Tools<div class="separator" style="clear: both; text-align: center;"><a href="http://2.bp.blogspot.com/-XV92N9NAPKg/Szu6GJR2B8I/AAAAAAAAAwc/bm5hTSaugBg/s1600/eschatonevent.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="199" src="http://2.bp.blogspot.com/-XV92N9NAPKg/Szu6GJR2B8I/AAAAAAAAAwc/bm5hTSaugBg/s200/eschatonevent.png" width="200" /></a></div>I'm releasing the code and docs that I wrote during my Xbox 360 research.<br /><br />py360 is a FUSE filesystem driver and associated file parsers for reading Xbox 360 hard drives. It is designed to aid forensic&nbsp;examination of the Xbox 360. The main components are a FUSE file system for mounting the XTAF file system and a set of python classes for parsing the the STFS, XDBF and Account block file formats and the&nbsp;XTAF file system.<br /><br />py360 is available from <a href="https://github.com/arkem/py360">github</a>&nbsp;and I've written a <a href="http://www.arkem.org/py360-user-guide.pdf">brief user guide</a>.<br /><br />Additionally the results of my look into these <a href="http://www.arkem.org/xbox360-file-reference.pdf">data structures is available</a>.<br /><br />This project would not have been possible without the existing work of the <a href="http://free60.org/Main_Page">Free60 project</a>.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com1tag:blogger.com,1999:blog-1404660204966454042.post-7924650261591180892011-03-24T02:17:00.002-07:002011-03-25T01:01:46.758-07:00Update!<a href="http://farm5.static.flickr.com/4029/4460960398_3307ea316c_o.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="320" src="http://farm5.static.flickr.com/4029/4460960398_3307ea316c_o.jpg" width="213" /></a>I'm still around, the Xbox 360 story will be completed soon!<br /><br />I've submitted my research, it has been accepted and I will graduate.<br /><br />I've been holding off on finishing the posts until I'd gotten it all submitted and I was in a place where I could post my results and some of my code. There may even be a google code project available at the end of all this.<br /><br />Patience please everyone!<br /><br />By the way, the image from the right is from&nbsp;<a href="http://www.trustocorp.com/">http://www.trustocorp.com/</a>&nbsp;who amuse me to no end. Go have a look.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-14986012008969382552011-01-28T02:22:00.000-08:002011-01-28T02:22:29.772-08:00Xbox 360 Forensics: Part 4<div class="separator" style="clear: both; text-align: center;"><a href="http://1.bp.blogspot.com/_Hn5k9rrcB9I/SzlePlu3QVI/AAAAAAAAAv8/cfzAFQPEAXw/s1600/wavylinesofdeath.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="200" src="http://1.bp.blogspot.com/_Hn5k9rrcB9I/SzlePlu3QVI/AAAAAAAAAv8/cfzAFQPEAXw/s200/wavylinesofdeath.png" width="200" /></a></div>In our last episode I had just about given up on x360 after spending a week or so digging around its code base, where did I go from there?<br /><br />Well I decided to sit down with the <a href="http://free60.org/">Free60.org</a>&nbsp;pages on&nbsp;<a href="http://free60.org/XTAF">XTAF</a>&nbsp;and on&nbsp;<a href="http://free60.org/File_System">Xbox 360 File Systems</a>,&nbsp;the source code of a couple of XTAF implementations (x360 and utaf.c), a hex editor and python and I started over. This was a fun project, it turns out that in my time patching x360 I'd learnt almost everything I needed to implement my own parser and some mucking around with a hex editor and the Free60.org docs got me the rest of the way. At that point I started reading about the&nbsp;<a href="http://en.wikipedia.org/wiki/Filesystem_in_Userspace">FUSE</a>&nbsp;bindings for python which are poorly documented but still make filesystem development super easy, I now want to implement everything as a filesystem!<br /><br />This was the beginnings of my py360 python module (which may one day see public release, bear with me!) and it was good. The first thing I did with my new filesystem and library was to take a recursive directory listing and perform a 'find . -type f -exec sha1sum {} \;' on the filesystem. When this went swimmingly I moved on to the next problem, which was to unleash libmagic (through file) on the filesystem but to my disappointment the mighty libmagic came back with nothing, everything showed as data. This was a problem and certainly not one that I had seriously anticipated. The next step was to see if I could make any sense out of these files quickly and the utility strings is my goto tool in these situations. Strings revealed that while there was a fair amount of ASCII text there was also a very large amount of UTF16 (big-endian) including what looked like the text of game achievements, unfortunately I couldn't find much of a reference to the gamertag of the user on this image nor any other stuff of interest. This is when I resigned myself to doing things the hard way... stay tuned!Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0