CUDA stack

A complete packaged CUDA stack has been added for all supported distributions. This now includes all CUDA libraries and tools at version 6.5.19 (includes NVML / GPU deployment kit). You can easily install CUDA 6.5 on CentOS/RHEL 6/7 and Fedora 20/21/rawhide!

All the packages provide/require/obsolete the relevant driver packages in the RPMFusion repository and all the CUDA packages in the Nvidia repository; so you can enable this repository along with the official Nvidia CUDA one and RPMFusion at the same time. Packages will get upgraded accordingly.

Nvidia is slowly fading out 32 bit support from CUDA, and you can see it reflected in the various packages. The Unified Video Memory kernel module (nvidia-uvm.ko has been removed in version 346.16, CUDA graphical programs are 64 bit only, many libraries and compilers are available in 64 bit only, etc.

Feedback from users has been integrated, where possible.

List of components by distribution:

Operating system

el6 / el7

f20 / f21 / f22

CUDA branch/version

6.5.19

6.5.19

Basic CUDA libraries/tools:

cuda
cuda-libs
cuda-extra-libs

Yes

Yes

CUDA development:

cuda-cli-tools
cuda-devel
cuda-docs.noarch

Yes

Yes

Java GUI programs:

cuda-nsight
cuda-nvvp

Yes

Yes

32 bit compatibility on x86_64:

cuda-devel
cuda-libs
cuda-extra-libs

Yes

Yes

Legacy drivers 340.xx

A compatibility repository for drivers on 340.x, the new legacy release for cards up to 9xxx chipsets has been introduced. It’s in the same place, just follow the instructions by appending -340 to the repository file. This repository does not include the CUDA packages, just the enablement on the drivers.

The repository itself it’s not guaranteed to stay online forever; the GTX 9xxx series are from 2008 and I don’t guarantee I will maintain it for long.

Delta RPMS

Delta RPMS have been introduced, to reduce the time and data required for upgrades. Driver packages can reach 90 mb and CUDA packages can span even 650 mb. This would save you a lot of time into upgrading them. For now, delta RPMS have been generated for the new 346.35 drivers, and this reduced nearly 80% the download size on Fedora 21.

We’ll see some real gain when updating the CUDA packages.

Ending words

Along this, there is the usual assortment of packages refinement (syntax, RPMLint, optimizations, etc.). For additional details, please see the Nvidia driver page.

As time permits, new CUDA enabled packages will be added to the repository, namely Blender, ccMiner, NVENC enabled ffmpeg, etc.

Like this:

Have you ever had the need to switch a CentOS system to a valid Red Hat Enterprise Linux subscription and viceversa?

I had this need quite a few times, with the most simple case being to transform an evaluation environment based on CentOS that has been used to convince a boss, into a fully supported Red Hat subscription at the end of the evaluation.

On the other hand, it could prove very useful to create an exact copy of an installed system that is currently attached to a Red Hat subscription on an image in your laptop for development purposes. Or simply because the Red Hat subscription has expired and we don’t need any kind of paid support from Red Hat.

As an example for this conversion tutorial, we’re using a CentOS/RHEL 6.6 system. The procedure is the same even if we are also switching minor release during the conversion; for example upgrading from 6.3 to 6.6.

From Red Hat Enterprise Linux to CentOS

This is the most simple case, mainly for two reasons. First of all, we can fetch the packages we need for the installation on the web (we don’t need a valid Red Hat subscription) and basically we are reducing the number of packages that are installed on the sytem in comparison with a pristine Red Hat Enterprise Linux system.

As the first step we must remove the -release package from the system and switch it with the one we’re interested in. This way the yum commands will be able to expand the necessary variables and fetch the correct packages.

To remove the leftover kernels on the system (including packages containing kernel headers, modules, etc.) we can run this command that will clean the system for us from all kernels except the one we’re running:

package-clenup -y--oldkernels--count=1

From CentOS to Red Hat Enterprise Linux

The opposite procedure is slightly different. It’s a bit simpler as the commands we require to type are reduced (quite a few packages from CentOS keep providing the old Red Hat package name in the Provides: tags. It’s otherwise a bit longer as we need a valid account to download by hand the required packages for the conversion; as the yum repositories (or channels) are not available.

After getting access to the Red Hat customer portal, we need to download the following packages to register the system (in our example we’re targeting a Server subscription):

No insecure LANMAN, NetBIOS, SMB1 enabled. Security is higher, performance is much better!

This makes the installation much hardened and secure than the default Microsoft setup.

This is an update to my old post for the new stuff that has changed during the past year and the introduction of CentOS/RHEL 7.

System enablement

This guide is written against Samba 4.x for Fedora and CentOS/RHEL 7 and a minimum Samba version of 4.1.6, as it’s the first Samba release that includes systemd support. You can grab the latest Samba source packages from Koji.

Both require a patched Samba package to enable the missing Domain Controller functionality. Hopefully this change will make it into official packages when Samba will be built with the system’s MIT Kerberos implementation.

The first patch is for disabling MIT Kerberos integration and enabling optional Heimdal Kerberos with Domain Controller functionality in the Redhat/Fedora package. This has also been reported upstream:

Software installation

Install BIND server (required also for other optional domains), the NTP server, the Samba suite (the rpms you just rebuilt) and some additional tools used by our environment on the selected server. For servers; replace also firewalld with the base iptables service:

Firewall configuration

The following ports need to be opened on the server firewall:

TCP: 53, 88, 135, 445, 464, 1024-5000

UDP: 53, 88, 123, 389, 464

Ports 1024-5000 are for the RPC services used by Samba, and can be further reduced in case you don’t have many clients. Port tcp/53 is used by Bind to receive DNS GSS record updates (they use TCP, not UDP). It is also used for large zone transfers, but this is not our case.

Create the file /etc/sysconfig/iptables and insert the following contents (we are assuming the server has an IP address of 192.168.0.17):

Provisioning the domain

First setup and provisioning can be executed with SELinux disabled and then later re-enabled. This helps debugging issues that are not otherwise present with DAC permissions. Since the domain controller functionality has not been enabled yet in the official packages; SELinux policies have not been updated yet. Execute the following commands as root to start the provisioning:

Disable NetBIOS

Edit the file /etc/samba/smb.conf and make sure that the [global] section contains the following lines (in addition to the others) to disable NetBIOS support:

[global]
server services = -dns, -nbt
smb ports = 445

When requesting a resource, Windows 2000 and later systems start two connections simultaneously to a server. One is on port 445 and one on port 139. If the client gets a response from port 445 it will reset (RST) the connection on port 139. If it only gets a response from port 139, that one is used. If you disable NBT (NetBIOS over TCP/IP) on your client; only port 445 is being tried. Pre-Windows 2000 clients (such as windows NT) only use port 139.

Configure Kerberos

Copy the provision generated Kerberos file to the default system location:

cp/var/lib/samba/private/krb5.conf /etc/krb5.conf

Make sure the Kerberos configuration file contains the check-ticket-addresses directive; as it is required for clients connecting through a NAT.

Change permissions of the NTP folders which should be accessible by the daemon:

chgrp ntp /var/lib/samba/ntp_signd/

Configure DNS server

Look at the hints in the previous provisioning output regarding BIND and modify the file /etc/named.conf. Remember to fill appropriately the zone files with the correct records. Replace my addresses with yours, of course.

If you disabled IPv6 for the system, disable IPv6 as well for BIND, this prevents flooding the logs with unwanted messages. Add the following line to /etc/sysconfig/named:

OPTIONS="-4"

Starting services

Make the Samba system use its Bind recursive DNS server as primary DNS. This is required for proper Samba 4 operation of the Domain Controller. Any external request made by the server will be forwarded through the POP DNS servers.

Edit /etc/sysconfig/network-scripts/ifcfg- and change the DNS1 line to read as follows:

DNS1=192.168.0.17

Then delete all other DNS* lines from the file. Afterwards restart the network:

systemctl restart NetworkManager

Finally start Bind, NTP server and Samba:

systemctl start named
systemctl start samba
systemctl start ntpd

Troubleshooting

For debugging, launch Bind, the NTP server and Samba with the following options to start them in the foreground:

The output identifies the last succesful sync time; the fact that the client / server communication is using MS-SNTP to communicate (Time Source Flags: 2 (Authenticated )), and that the last command was executed successfully.

In case it doesn’t work; to manually set Windows Time Service configuration to read NTP settings from the domain, perform the following commands to reset the configuration and to sync again the client to the server:

w32tm /config /update /syncfromflags:DOMHIER
w32tm /resync

Then check again the status with the previous command.

If the time server specified in the Windows client is a normal NTP server, then the Windows client will not ask for MS-SNTP signed responses. The command to synchronize the clock will be as follows:

DNS and GSSEC records insertion/deletion

To test DNS dynamic updates perform the following command on the Windows client:

ipconfig /registerdns

This will create a DNS record for the system in the Active Directory DNS zone using a secure Kerberos authenticated update.

If the record does not appear; start debugging on the server for DNS records availability and proper functioning of the DLZ zone. To proceed launch the following command with both Samba and Bind running:

samba_dnsupdate --verbose--all-names
samba_dnsupdate --verbose

This will fetch all the minimum required DNS records for Active Directory from the Samba database and try to re-insert them into the zone using a kerberized (GSSEC) DNS update to the Bind server.

In case you obtain the message dns_tkey_negotiategss: TKEY is unacceptable while trying to run the command; tis means you have some problems with your current Bind Kerberos keytab file. Perform the following command to check that the service principals are contained in the file:

Windows client networking adjustments

Disable NetBios over TCP/IP

To make the necessary tests; make sure that the Windows system has NetBIOS over TCP/IP disabled in the Advanced TCP/IP settings configuration pane.

When requesting a resource, Windows 2000 and later systems start two connections simultaneously to a server. One is on port 445 and one on port 139. If the client gets a response from port 445 it will reset (RST) the connection on port 139. If it only gets a response from port 139, that one is used. If you disable NBT (NetBIOS over TCP/IP) on your client; only port 445 is being tried. Pre-Windows 2000 clients (such as windows NT) only use port 139.

Disable Teredo IPv6 Tunneling

To disable IPv6 and Teredo IPv6 Tunnelling execute the following command as an Administrator in the Windows command prompt:

netsh interface teredo set state disabled
netsh int ipv6 isatap set state disabled
netsh int ipv6 6to4 set state disabled

Windows firewall integration

For Windows 7, the following ports need to be enabled in the firewall; all the other rules should be disabled. This is a subset of the ones listed in Microsoft’s Active Directory required ports:

Communications from the Windows client towards the domain controller:

TCP: 135, 445, 3268, 1024-5000

TCP/UDP: 53, 123, 88, 389, 464

Communications from the domain controller towards the Windows clients:

TCP: 135, 445, 1024-1048

TLS support for LDAP (local domain on 389 and Global Catalogue on 3268) is disabled because connections are made with SASL, using GSS-API and thus employing Kerberos and session-level encryption. For details on message integrity (signing) and message confidentiality (sealing) please see this nice article from the University of Washington that explains authentication in a simple way.

RPC ports can be as low as one, but in this case you lose a lot of the functionality. For example, running a scheduled task on Windows will open an additional RPC port (yes, that’s true), and if the system does not have any one that can be used, the process fails miserably. From my test in our office, 24 ports should be enough for domain management plus normal day to day desktop use.

To make the RPC server listen on port range 1024-1048; the following registry file needs to be applied and the system rebooted:

Share this:

Like this:

I’ve created experimental CUDA packages that try to follow Fedora packaging guidelines as close as possible. Those have been updated to the Nvidia Fedora 20 repository, and are installable through normal yum commands.

To install them, you need to use my repository that contains the latest drivers.

32 bit support

Nvidia is slowly fading out 32 bit support from CUDA, and you can see it reflected in the various packages. The Unified Video Memory kernel module (nvidia-uvm.ko has been removed in version 346.16, CUDA graphical programs are 64 bit only, many libraries and compilers are available in 64 bit only, etc.

Package testing

I’ve uploaded packages only to the Fedora 20 repository, as they are very big and this is what I’m using at the moment as my main desktop. To help test these, I’ve also added a package for ccminer, a CUDA cryptocurrency miner that links to the packages and requires them to be built. By installing it, all required CUDA runtime libraries should be installed as well.

If all goes well, my plan is to enable CUDA packages for all supported Fedora/CentOS/RHEL distributions and add also package software that in the current form do not use the Nvidia libraries.

As an example, the Blender package in Fedora does not (obviously) link to the CUDA libraries, so no CUDA rendering. On the contrary, the binary that you can download from the Blender website is linked statically to the CUDA libraries at compile time.

After some feedback I will enable them for all the other distributions. So if you need them, please test them.

Packages available

A brief recap on the packages, here we have the full list of drivers and CUDA packages that are available inside the repository folder:

Package bundles

From the above list, packages can be grouped as follows for an x86_64 system. For additional details, please see the repository page.

Kernel modules, in both akmod and dkms variants. Instantiated kernel modules are available as rebuild in both by enabling the appropriate configuration on your system.

akmod-nvidia
dkms-nvidia
kmod-nvidia

These are the basic driver packages, they are what is required along the kernel module packages to have accelerated drivers and full OpenGL support for a normal desktop. That is gaming, office use, etc. But no CUDA support.

Then there are extra tools and libraries, like Framebuffer Compression OpenGL libraries, GPU Deployment Kit (NVML, also called Nvidia Management Library), command line configuration for very specific X.org setups and a tool that leverages the NVML library to perform health checks on GPU clusters.

Please note that the Nvidia Management Library headers and tools do not follow the same versioning of the main driver set as they are provided by Nvidia in a separate bundle that is compatible across multiple releases of the drivers. For example, at the time of writing this, we have 340.58 (long lived), 343.22 (short lived) and 346.18 (beta) drivers available.

After those, that have been provided here for more than a year, I’ve now added CUDA packages. These can be splitted into multiple components as well; first group contains most runtime components for simply running CUDA enabled programs:

cuda-libs.i686
cuda-libs
cuda-extra-libs.i686
cuda-extra-libs

Then we have development files (headers, stub libraries, documentation, compilers, etc.) for compiling programs that link to the CUDA libraries:

cuda
cuda-cli-tools
cuda-devel.i686
cuda-devel
cuda-docs.noarch

And then finally, we have Java GUI programs (debuggers, etc.):

cuda-nsight
cuda-nvvp

Official Nvidia repositories

All the packages provide/require/obsolete the relevant driver packages in the RPMFusion repository and all the CUDA packages in the Nvidia repository; so you can enable this repository along with the official Nvidia CUDA one and RPMFusion at the same time. Packages will get upgraded accordingly.

There are test builds available, if you happen to use DKMS please leave some feedback. These need some testing on all supported distributions (that is RHEL/CentOS 5, 6 and 7; and Fedora 19, 20, 21), so it will stay in the testing repositories for quite some time until I get some feedback.

Share this:

Like this:

Just came back from holiday and work travel; just in time for another batch of changes for the repositories:

MakeMKV has been updated to version 1.8.12.

HandBrake has been updated to the current development version for Fedora 20, 21 and 22. It enables now x265 by default and uses even less bundled libraries. It also uses the system libappindicator for notifications.

Flash plugin package has been updated to version 11.2.202.400.

Spotify it’s now at version 0.9.11.27.g2b1a638.81-1. As announced previously in another post, the original i386 build was reverted by upstream to 0.9.4 without even notifying and it stayed locked at this version for months. As such, I completely removed i386 support for it. I’ve changed the repository page to reflect it.

The Nvidia Management library header and man pages (alias GPU Deployment Kit) it’s now at version 340.21, the bundled version in CUDA 6.5. It works with drivers version 340.x and up.

Most of the repositories now support both Fedora 21 and 22, including HandBrake & friends.

Also, I’m building CUDA 6.5 packages now that it has been released. Testers welcome as usual.

Like this:

The HandBrake repository has seen some updates; it now contains the current development version for Fedora 20 and later, based on feedback in the previous weeks. It contains less and less bundled libraries and now uses the GTK3 interface.

libdvdnav is now on the final 5.0.0 release. Not much changed since the previous release that contained all the fixes required to avoid the HandBrake crash while opening a DVD for scanning with the default settings.

The Steam package no longer ships a steam-noruntime subpackage, Fedora’s current system libraries diverge too much from the bundled Ubuntu runtime, and Steam refuses to start with them. Most of the games still run fine though, and nothing prevents you from running them without the Ubuntu libraries.

The package now obsoletes steam-noruntime, and it has also been pushed to RPMFusion.

Other Steam updates are for the SteamOS packages, they are now on par with the SteamOS update 126.

Quite a few things have changed, the packages are going towards providing the complete packaged CUDA stack. So far, only the GPU deployment kit has been inclued; and the packages allow for parallel installation with the Nvidia CUDA repositories by osboleting/updating packages as required. Here is some details on the things that have been implemented.

X.org configuration

Starting from Fedora 21, all driver X.org configuration can be managed by simply adding/removing X.org configuration snippets in /etc/X11/xorg.conf.d.

Use new OutputClass directive on Fedora 21 X.org server 1.16 (and later) to load the driver and do not rely on an edited /etc/X11/xorg.conf file. This also removes editing of the xorg.conf file from the package scriptlets.

Add the IgnoreABI directive by default on Fedora rawhide builds.

Kernel modules

Add a new UDev rule in nvidia-driver-cuda for the nvidia-uvm module and make X.org NVIDIA Files section to be loaded latest in case there are other packages providing a custom Files section (thanks Jan P. Springer for spotting these).

The binary nvidia-modprobe is now SETUID, but its package is no longer a mandatory requirement for the drivers, so it will not get installed by default.

Now that both Redhat Enterprise Linux 7 and CentOS 7 have been released, binary modules (kABI) are now provided for these distributions.

CUDA support

Added the GPU Deployment kit to the repository. This is constructed with NVML (NVIDIA Management Library) included with the drivers plus headers, docs and samples from a separate tarball. The separate tarball is using a different version number than the drivers. This is packaged in the nvidia-driver-NVML and nvidia-driver-NVML-devel packages. Installing these, the gpu-deployment-kit dependency provided by the CUDA repositories is preserved.

Along with NVML, the nvidia-healthmon package is provided to monitor TESLA GPU clusters.

Along this, there is the usual assortment of packages refinement (syntax, RPMLint, optimizations, etc.). For additional details, please see the Nvidia driver page.

If you would like to test the CUDA packages please contact me and I will point you to a repository hosting the CUDA packages.