Hi,
I’ve found this discussion https://lists.01.org/pipermail/spdk/2016-December/000251.html
on memory allocation. I have similar situation – our application uses some allocator to
create IO buffers, but of course, SPDK would not accept such a buffer. So I have two
questions:
1) I cant find in code the spdk_vtophys_register() mentioned in the above link, is it
removed? If not, where is it? How to use it?
2) In case our allocator uses hugepages and it is aligned on 4Kib boundary, is there a way
to make the SPDK believe that this buffer could be used for NVMe IO?
Sincerely,
E.

From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Ernest Zaslavsky
<kreuzerkrieg(a)gmail.com&gt;
Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Date: Tuesday, February 6, 2018 at 6:32 AM
To: &quot;spdk(a)lists.01.org&quot; <spdk(a)lists.01.org&gt;
Subject: [SPDK] Memory buffer allocation in SPDK
Hi,
I’ve found this discussion https://lists.01.org/pipermail/spdk/2016-December/000251.html
on memory allocation. I have similar situation – our application uses some allocator to
create IO buffers, but of course, SPDK would not accept such a buffer. So I have two
questions:
1) I cant find in code the spdk_vtophys_register() mentioned in the above link, is
it removed? If not, where is it? How to use it?
Hi Ernest,
spdk_mem_register() is the equivalent function now. This will register the specified
buffer not only for virtual-to-physical address translation, but also for RDMA operations
using the SPDK NVMe-oF driver.
2) In case our allocator uses hugepages and it is aligned on 4Kib boundary, is there
a way to make the SPDK believe that this buffer could be used for NVMe IO?
spdk_mem_register() is the function to use. Note that currently for vtophys translation,
the specified buffer must be 2MiB aligned and an even 2MiB multiple.
Regards,
-Jim

Hi James,
Thanks for the info, sounds like we have a way to solve our problem. I have some
additional questions. The spdk_mem_register() function, it just adds the region to the
SPDK allocation map, right? It does not touch or modifies the memory in any way, right? Is
it safe to say that we may register with SPDK all our pre-allocated memory when the
application starts and just keep it that way?
Sincerely,
E.
From: Harris, James R
Sent: Wednesday, February 7, 2018 2:11 AM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Memory buffer allocation in SPDK
From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Ernest Zaslavsky
<kreuzerkrieg(a)gmail.com&gt;
Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Date: Tuesday, February 6, 2018 at 6:32 AM
To: &quot;spdk(a)lists.01.org&quot; <spdk(a)lists.01.org&gt;
Subject: [SPDK] Memory buffer allocation in SPDK
Hi,
I’ve found this discussion https://lists.01.org/pipermail/spdk/2016-December/000251.html
on memory allocation. I have similar situation – our application uses some allocator to
create IO buffers, but of course, SPDK would not accept such a buffer. So I have two
questions:
1) I cant find in code the spdk_vtophys_register() mentioned in the above link, is it
removed? If not, where is it? How to use it?
Hi Ernest,
spdk_mem_register() is the equivalent function now. This will register the specified
buffer not only for virtual-to-physical address translation, but also for RDMA operations
using the SPDK NVMe-oF driver.
2) In case our allocator uses hugepages and it is aligned on 4Kib boundary, is there a way
to make the SPDK believe that this buffer could be used for NVMe IO?
spdk_mem_register() is the function to use. Note that currently for vtophys translation,
the specified buffer must be 2MiB aligned and an even 2MiB multiple.
Regards,
-Jim

From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Ernest Zaslavsky
<kreuzerkrieg(a)gmail.com&gt;
Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Date: Tuesday, February 6, 2018 at 10:51 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Subject: Re: [SPDK] Memory buffer allocation in SPDK
Hi James,
Thanks for the info, sounds like we have a way to solve our problem. I have some
additional questions. The spdk_mem_register() function, it just adds the region to the
SPDK allocation map, right? It does not touch or modifies the memory in any way, right? Is
it safe to say that we may register with SPDK all our pre-allocated memory when the
application starts and just keep it that way?
Hi Ernest,
Correct – SPDK does not touch the buffer registered with spdk_mem_register() in any way –
it only adds it to any registered translation maps.
-Jim

Great! Thanks!
Sent from Mail for Windows 10
From: Harris, James R
Sent: Wednesday, February 7, 2018 5:00 PM
To: Storage Performance Development Kit
Subject: Re: [SPDK] Memory buffer allocation in SPDK
From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Ernest Zaslavsky
<kreuzerkrieg(a)gmail.com&gt;
Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Date: Tuesday, February 6, 2018 at 10:51 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org&gt;
Subject: Re: [SPDK] Memory buffer allocation in SPDK
Hi James,
Thanks for the info, sounds like we have a way to solve our problem. I have some
additional questions. The spdk_mem_register() function, it just adds the region to the
SPDK allocation map, right? It does not touch or modifies the memory in any way, right? Is
it safe to say that we may register with SPDK all our pre-allocated memory when the
application starts and just keep it that way?
Hi Ernest,
Correct – SPDK does not touch the buffer registered with spdk_mem_register() in any way –
it only adds it to any registered translation maps.
-Jim