What I have is a genuinely NTFS formatted 4TB drive. (The drive was formatted in Windows 8, fwiw). It's an external USB drive (Western Digital).

What I would like to do is to mount it and access it from OpenBSD. (Yes, I realize it's only 'read only' and this is not a problem!)

What happens is that I get "invalid argument" when I try to mount it using mount_ntfs. I have tried formatting it a few different ways and still the same result. What am I missing? I am still very much an openbsd beginner/ hobbyist..

When I saw this, I thought, "uh oh". That's because MBR partition tables are limited to 2TB, and disks larger appear in the MBR as 2TB. As mentioned in FAQ 14.8, this is not a problem for OpenBSD, as the OpenBSD disklabel doesn't have this restriction. However, you are using a disk that contains only a foreign filesystem, and then a "virtual" disklabel based on the MBR, which is of course limited to 2TB.

Setting the right partition size: I recommend creating a physical disklabel on the drive, so that OpenBSD can address the actual size of the NTFS partition. e.g.: create a small OpenBSD MBR partition of type A6 on the drive, to house the disklabel. This can be a very small partition, such as 1MB in size. Then use disklabel(8) to correctly map the "real" NTFS partition by starting sector and size into the physical disklabel.

Sector size may be an issue: While it may seem simple to humans to use a larger sector size, since the 1970s fixed block architecture drives have been using 512-byte sectors, and 4096-byte sectors are "brand new". There's a lot of OS and driver code that does not support 4096-byte sector drives. Therefore, I would not be surprised if the built-in NTFS driver in OpenBSD might not support 4096-byte sector devices.You should test this with a small NTFS partition, to determine if this is a problem before resolving the 2TB limitation imposed by MBR partitioning. In the event that the built-in NTFS driver does not support 4096-byte sectors, then you might consider trying the ntfs_3g package, which uses OpenBSD's FUSE implementation. Along with the possibility of supporting 4096-byte sectors, this NTFS driver also supports write access.

I think you may have two separate issues.When I saw this, I thought, "uh oh". That's because MBR partition tables are limited to 2TB, and disks larger appear in the MBR as 2TB. As mentioned in FAQ 14.8, this is not a problem for OpenBSD, as the OpenBSD disklabel doesn't have this restriction. However, you are using a disk that contains only a foreign filesystem, and then a "virtual" disklabel based on the MBR, which is of course limited to 2TB.

Setting the right partition size: I recommend creating a physical disklabel on the drive, so that OpenBSD can address the actual size of the NTFS partition. e.g.: create a small OpenBSD MBR partition of type A6 on the drive, to house the disklabel. This can be a very small partition, such as 1MB in size. Then use disklabel(8) to correctly map the "real" NTFS partition by starting sector and size into the physical disklabel.

Hay thank you so much!

OK I tested using an old half terabyte drive i had lying around, 4K sectors formatted by Windows do seem to work. I do not think I was confused enough last time to format some sort of dynamic disk.

Then I will bark up the next tree. First thing I am rather naively wondering is, if I can fit this OpenBSD MBR partition you are suggesting into the 63 sectors before the NTFS partition starts. Well.... perhaps it would be wise to screw around with this on my "old drive I had lying around" first ..

Actually, come to think of it, first I will just carefully read the manual entry for OpenBSD's fdisk.

( and yes, /mnt/ntfs does exist and I was just able to mount the above mentioned test NTFS volume on it OK )

The disklabel's sector location on a drive (or, if there is an MBR partition, within the MBR partition) is architecture specific, defined in /usr/include/machine/disklabel.h

While I think there should be "room" in the first sectors, the first track is reserved for both the MBR and for additional boot blocks, so you may be better off adding a partition entry for OpenBSD -- I'd recommended a megabyte, or, perhaps you can reserve just the second logical track.

The goal is to avoid overlaying anything important: either a boot block, or the disklabel.

Again thanks for the ideas, I had some time to kill and tried .. something (see below), but it still doesn't work. There is a OpenBSD disklabel and a even a rather small ffs partition. Mount (mount_msdos i guess) works on a FAT32 partition but mount_ntfs does not, on the NTFS one.

I reckon i will bark up the ntfs_3g tree next, if nothing else I'm sure I'm not the only one in the world stumped by this and it could be useful for the solution (or a conclusion of an impossibility thereof) to be googlable.

You would indeed think, but somehow it was not. I just plugged the drive into a windows machine and the drive just appeared as usual, with any files intact. I also ran a filesystem check on this volume from within windows and it's clean ('Disk check complete'). I am not sure why newfs indicates 328 sectors of 4096 bytes (comes to about 1.28 mb) when df -h says 128kb on the mounted file system (which is a little more consistent with the 48 sectors * 4096 bytes = 192 kb I thought i had allocated)

I had to add the '-b 4096' to even get newfs to make such a puny filesystem. I'm sure it's fairly uncommon that anyone would make a 192 kb filesystem on a drive with 4096 byte sectors..

Here's a stupid question. What release of OpenBSD are you using? NTFS was not added to the GENERIC kernel until OpenBSD 4.9. Prior to that release, NTFS requires a custom kernel. And, NTFS is only available for the i386 and amd64 architectures. Here's the commit:

For many years, NTFS support (via a combination of a kernel driver and the mount_ntfs command) were considered experimental, and only read/only access was supported. The mount command was always present, but the admin had to configure and build a custom, unsupported kernel.

With 4.9, NTFS was added to the i386 and amd64 kernels by default. Because this no longer required a custom kernel, the status of NTFS, still read-only, went from experimental to supported.

The comment in the commit was in regards to the future possibility of revising the driver to permit read/write access. This did not come to pass. However, FUSE support was added (intially and experimentally) to OpenBSD at 5.4, and the ntfs_3g FUSE driver was added to the ports tree.

There have been manysignificant improvements to FUSE support after 5.4. Along with functional improvements since, FUSE was experimental with 5.4, and it would require both a custom kernel and a custom library build in order to enable its operation.

If you are considering using ntfs_3g, you should upgrade to at least 5.5-release.

I added the package ntfs_3g-2013.1.13p2 . Unfortunately, still no dice. Well, at least I get a little bit more (googlable) information from the error message (and it does seem that ntfs-3g gets a bit more tender and loving care overall than the stock openbsd ntfs code).

Plus hey, I am running 5.5 instead of 5.4, and if I get it working I suppose I could also potentially try writing to a ntfs volume... well, the quest continues!

Now I am quite sure there is something fundamental I am missing. I googled around a bit and used ktrace/kdump on the utility ntfsinfo (which produces a similar error). It turned out the point of failure is a call to pread() to get the boot sector, which is where the 'invalid argument' error is returned from. Now I realize I can't even access the first sector as root using dd either:

Wrong block size with dd. Your sectors are 4096 bytes long, not 512 bytes. My supposition at the top of this thread related to this very thing.

Yes, you are indeed on to something here. With block size 4096, dd still fails on sd0c but succeeds on rsd0c. With block size 512 it fails on either. (Perhaps this is not surprising to you, but it is to me, I still thought dd could read partial sectors from anything)

The '-m' arguments just means 'read and display the metadata for the ntfs volume'. I checked, ntfsinfo also fails in the same way on rsd0i. If you notice the dump above, it tries to read 0x200 = 512 bytes from file descriptor 3, which is /dev/sd0i :

... or could it be some weird memory alignment issue, that becomes evident here but not so much under Linux, because perhaps OpenBSD handles RAM itself differently?

Eg in the dump from the failed ntfsinfo execution I notice that 0x7bff0a00 (which I imagine is the address in RAM of the buffer for pread() to store the data) just happens to be divisible by 512, but not by 4096.

Thanks for doing the digging. I happened upon this comment at Tuxera just now.The version with 5.5-release is 2013.1.13, so either short-sector reads are OK on other OSes, which is possible, or some sector sizes were still hard-coded in 2013.1.13.

OpenBSD-5.6 will use version 2014.2.15. I haven't looked at the code for either. Again, if I have a few hours I will try to replicate the problem this weekend.

OK. Yes indeed, 2013.1.13 is what it seems to be. I still don't understand why also dd fails with a block length shorter than the physical blocks. For example, with even shorter blocks (256 bytes long), dd works on a 512 bytes/sector disk (wd0c) but fails on the 4096 bytes/sector one (sd0c). Same system.. there really is something I am not understanding here.. either there is some mystical interaction between the different software parts between the actual disk and /dev/sd0, [uhub- umass- scsibus- whatnot], or this is the standard defined behavior of dd when you use smaller block size than physical size. (Also on a small linux machine (raspberrypi) dd seems to be OK with 256 bytes long blocks .. )