Steve's blog about RMPrepUSB, Easy2Boot and USB booting. Please tick one of the 'Reactions' boxes if you enjoyed a post.
Visit www.rmprepusb.com for over 130 Tutorials on USB booting or www.easy2boot.com for a unique USB multiboot solution.

Wednesday, 2 October 2013

A faster test for 'fake' SD cards and USB Flash drives!

Please note: FakeFlashTest DOES NOT CONTAIN ANY VIRUSES, even if Windows Defender tells you it does!

Why should I use FakeFlashTest?

If you have ever bought an SD card or USB Flash drive, I hope you tested it first before you used it. A good video about how this can affect you is here. You are probably aware (or maybe not if you are reading this because you have just bought one!) that there are millionsof these fakeFlash memory devices being sold. Some sellers on eBay even admit (in the small print) that they are 'upgraded' and may not actually contain the amount of memory you paid for, yet eBay still continue to allow these people to sell them!

Here is an extract from the description for a 64GB SD card that is currently being sold on eBay as 'good'...

Please read below before you commit to bid on or buy one of these cards .

If you bid on or buy one of these memory cards it is of the understanding that you have read, understood and agreed with the below. These brand new upgraded/expanded micro SD SDHC memory cards have been tested and marked as 64GB and are all listed as good and are upgraded/expanded from 8GB to 64GB.These are the same as pretty much all cheaper memory cards on eBay; they just don't tell you this.Please note that while these cards are brand new and have been shipped to me straight from the factory these are sold as upgraded /expanded memory cards and therefore the speed may vary for each card. (They may experience read/write errors once max capacity reached) These items are approximately 1/3 of what you would pay for brand new brand named perfect condition products hence the very low price! Once you receive your card it is always recommended that you format it in your device before use. We also recommend that you back up your data from time to time.We cannot be held responsible for any loss of data that has not been backed up after use of this card *These cards will fail the h2testw test*More eBay examples here and here (this seller advertises them as 128GB and then says in the small print that they are only 16GB!!!).

So these cards are 'listed as good and are 'upgraded/expanded' ??? No! - they are fake and have been 'clocked' deliberately to deceive you. Furthermore, any normal method of formatting will cause them to be formatted as 64GB cards and so you will lose any files stored past 8GB when you use them. In fact, you are paying 1/3rd of the price of a real 64GB card for 1/8th of the capacity (8GB) and it is of dubious quality and you will lose all your precious files and photos if you try to use it!

They have been deliberately re-programmed to lie to you!

The seller seems to think that it is normal for all cheap SD cards to be fake! It shows just how widespread the problem is. These devices are not 'faulty' per se, they have been deliberately re-programmed to lie about their size. For instance a 64GB SD card may actually contain a 8GB Flash memory chip but Windows or your camera or smartphone will detect it as a 64GB device and even successfully format it as a 64GB device (though often, if you format it as NTFS, it may fail to format!). It's like sticking a Ferrari badge on a VW Beatle and selling it as a Ferrari for 1/3 of the price of a Ferrari and saying everyone does it so it's OK!

These devices usually work for any reads or writes of the first 8GB to the memory chip inside it, but any writes made to locations past 8GB, will either overwrite the data already in the 8GB chip(s), or the controller inside the device will simply not write the data at all, but signal back to your computer/camera/smartphone that the write worked fine! As the directory and cluster map of a FAT32 file system is stored within the first few 100MBs, the filenames will be listed by Windows Explorer or your camera just fine (until you try to look at that data).

Here is an interesting eBay advert of an honest seller who mistakenly sold fake 2TB devices and then found out they were fake.Testing Flash memory
There are several utilities that are suggested for testing these devices, the 'Gold Standard' one being H2TESTWv1.4 written for ctmagazin.de (however, I am not sure why this is the 'Gold Standard' as it does not test the area already occupied by your files). There are also other flash test utilities such as chkflsh and USB Flash Drive Tester.

The problem with these utilities is they attempt to write and read to all the bytes in the memory (or in the case of H2TESTW, to all the unused filespace). In 2008 when H2TESTW was last revised, they had 256MB flash sticks and you needed to be rich to afford an 8GB device. Today however, we have 64GB 'fake' SD cards and these can take hours to test using H2TESTW and chkflsh, as these tests 'fill it all up' and then 'read it all back' - every single byte; and only then gives a report. This is totally unnecessary for the purposes of checking for fake flash memory because we will easily see gross errors - whole blocks will be missing or corrupt. So there is no need to write all 64GB and then read all 64GB back again - with a little intelligence we can greatly speed this up.

RMPrepUSB

RMPrepUSB was a Windows USB drive preparation utility which I wrote several years ago.

The Quick Size Test in RMPrepUSB was designed to check the size of these flash memory devices quickly. It does this by only writing 512-bytes sectors at various intervals to the memory and then reading back these sectors and checking them. In this way we only need to write and read to a fraction of the sectors in the device. Consequently it is much quicker than any other test! However, the RMPrepUSB Quick Size Test is destructive - the memory 'blocks' are tested, but in order to do that, they have to be overwritten.

Also, the multi-button, multi-tab RMPrepUSB GUI is a little overwhelming to the average user who just wants to check their new SD card or USB Flash drive.

FakeFlashTest - A more user-friendly test

An RMPrepUSB user suggested that I make a single app just for fake flash testing, so I wrote FakeFlashTest.exe and used the same Quick Size Test code in FakeFlashTest (FFT).

Quick Size Test algorithm

512-byte blocks of unique data are written at large intervals to the entire drive, starting from the end of the drive. The blocks are then read back, starting from the beginning of the drive.

Only a fraction of the total blocks in the drive are actually written to, so the test is much quicker that H2TESTW.

Non-destructive (slower) algorithm

I also wanted to add a second test to FFT, similar to the technique used by H2TESTW (non-destructive) - i.e. write files to fill up the device volume and then read back the data in all the files and check it. However, I could do several things differently in order to speed up this test process:

1. Don't generate a random unique pattern for every file. I use A5h bytes for the basic pattern but then write a unique number at intervals inside the buffer. The unique number is an id marker for that file.
2. The drive is filled by files of 200MB in size, by using this buffer (but with the id bytes changed for each file)
3. When the files are read back, I only read back the marker id bytes and use a file seek operation to get the next marker position. This saves having to read the entire file back and verify all the data.
4. Most fake flash memory devices will contain 1, 2, 4 or 8GB of memory - the rest of the memory will be non-existent. Therefore it is pointless to write 64GB of data in one go and then read all 64GB back again, when we probably corrupted the first 4GB as soon as we wrote just over 4GB of data to it. For this reason, after I write 3GB of files, I verify the 3GB to see if they are corrupt. I also do the same after writing 5GB and after writing 9GB and 17GB. After 17GB, I then don't verify any of the files until I have completely filled the drive volume.
5. If I hit a gross error during the file write process, then execution is stopped. After all, we have our answer - the device is faulty, no need to wait 8 hours! However, if all files were written without error, I check all the files and can then report the amount of usable, uncorrupted space. Note: For this reason the reported 'true capacity' returned by this 'Empty Space' test may be grossly incorrect!

YouTube video about FakeFlashTest and Fake Flash memory.

FFT has two main buttons.

Quick Size Test - The large button is for the RMPrepUSB style, destructive but very fast Quick Size Test. I recommend you use this test to test all your SD cards and USB Flash drives and then reformat them (note: RMPrepUSB improves file write speed on a FAT32 volume by up to 10% compared to other utilities!).

Test empty Space Test - The smaller button runs an H2TESTW-style file fill test but with the enhancements outlined above. Be aware that even though this just adds files to the drive, if your drive is a 'fake' or faulty then you may still lose ALL your files! Also note that if an error is found, it just stops testing and reports a failure (to save a lot of time), so the estimated 'good' capacity will not be at all accurate when doing this 'file fill' test. For a more accurate size test, use the Quick Size Test button.

So, how does it perform? Here are the results using a fake 16GB USB flash drive, formatted as an empty 16GB FAT32 volume, which actually appears to have only 2GB of memory in it.

H2TESTW - 48:40 minutes!!!

FFT Empty Space (file) test - 7:17 minutes

FFT Quick Test - 27 seconds + estimate of 'good' memory available!

Time differences for a 'good' USB stick are not as impressive because I used a USB 3.0 UFD in a fast USB 3.0 port and so data I/O was pretty fast anyway. A slower USB 2.0 drive would show a much larger difference when compared to H2TESTW. H2TESTW FFT-Files test FFT-Quick test8GB GOOD (USB 3.0) 11:46 11:01 6:10 in USB 3 port16GB BAD (USB 2.0) 48:40 7:17 0:27 14GB had 10msec access!

P.S. Check you have the latest version of FFT (later versions work with non-USB card readers as well as USB card-readers).

What to do if you have a 'fake' SD card/flash drive
1. Complain to the seller and leave negative feedback (even if they fully refund you). They are cheats and frauds and are deceiving everyone. Someone else may go on a once-in-a-lifetime safari holiday to Africa or a Wedding or other important event, only to find that they have lost all their photos when they get back home! If it was purchased from eBay/Amazon then report them!

2. Try using RMPrepUSB to format the fake device to it's real capacity (as reported by FFT Quick Size test). Then test it using the FFT - Test Empty Space button or H2TESTW to check that all bytes are good. Even if you do this, there is no guarantee that the memory chips are reliable - many use reject memory chips. Mark the device so that you know it is a fake and write the real capacity on it. If you reformat it, you must reformat it to the real capacity of the memory actually fitted. Do not use your camera or smartphone to reformat it or it will format it to the maximum 'fake' capacity again. If you find a fake Flash device - please click on the 'Funny' button below to show other people just how many there are around!