tag:blogger.com,1999:blog-39500730894265069792017-08-12T08:39:20.904+02:00Sameh AttiaSameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.comBlogger1942125tag:blogger.com,1999:blog-3950073089426506979.post-21497378756399844622017-03-23T10:31:00.003+02:002017-03-23T10:31:49.891+02:00How To Find The Geolocation Of An IP Address From Commandline<div dir="ltr" style="text-align: left;" trbidi="on">https://www.ostechnix.com/find-geolocation-ip-address-commandline<br /><br /><div class="post-format"> <div class="image-container"> <img alt="Find The Geolocation Of An IP Address From Commandline" class="attachment-thumb-large size-thumb-large wp-post-image tc-smart-loaded" data-lazy-loaded="true" height="340" src="https://www.ostechnix.com/wp-content/plugins/lazy-load/images/1x1.trans.gif" style="display: block;" width="720" /> </div></div>A while ago, we wrote an article that described how to&nbsp;find out your geolocation from commandline using <a href="https://www.ostechnix.com/get-geolocation-commandline-linux/" target="_blank"><strong>whereami</strong></a> utility. Today, we will see how to find the geolocation of an IP address. Of course, you can see this details from a web browser. But, it is lot easier to find it from commandline.&nbsp;<strong>geoiplookup</strong> is a command line utility that can be used to find the Country that an IP address or hostname originates from. It&nbsp;uses the GeoIP library and database to collect the details of an IP address.<br /> <div style="float: none; margin: 10px 0 10px 0; text-align: center;"> </div>This brief guide describes how to install and use geoiplookup utility to find the location of an IP address in Unix-like operating systems.<br /> <h2>Find The Geolocation Of An IP Address Using Geoiplookup From Commandline</h2><h4>Install Geoiplookup</h4>Geoiplookup is available in the default repositories of most Linux operating systems.<br /> To install it on Arch Linux and its derivatives, run:<br /> <pre>sudo pacman -S&nbsp;geoip</pre>On&nbsp;Debian, Ubuntu, Linux Mint:<br /> <pre>sudo apt-get install geoip-bin</pre>On RHEL, CentOS, Fedora, Scientific Linux:<br /> <pre>sudo yum install geoip</pre>On SUSE/openSUSE:<br /> <pre>sudo zypper install geoip</pre><h4>Usage</h4>Once installed, you can find out any IP address’s geolocation like below.<br /> <pre>geoiplookup&nbsp;80.60.233.195</pre>The above command&nbsp;will find and display the Country that 80.60.233.195 originates from, in the following format:<br /> <pre>GeoIP Country Edition: NL, Netherlands</pre><div id="exam_announcement"><b>Download</b> – <a href="https://ostechnix.tradepub.com/free/w_annb07/prgm.cgi?a=1" style="text-decoration: underline;" target="_blank"><strong>Free eBook: “Computer Networking Concepts”</strong></a><span class="hu-external"></span></div><h4>Download and update Geoip databases</h4>Generally, the default location of Geoip databases is <strong>/usr/share/GeoIP/</strong>. The databases might be bit outdated. You can download the latest databases that contains the updated geolocation details, from <strong>Maxmind</strong>.&nbsp;It is the website that offers the geolocation of an IP address.<br /> Go to geoip default database folder:<br /> <pre>cd&nbsp;/usr/share/GeoIP/</pre>Download the latest databases:<br /> <pre>wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz</pre><pre>gunzip GeoIP.dat.gz</pre>Now, run the geoiplookup command to find most up-to-date geolocation details of an IP address.<br /> <pre>geoiplookup 216.58.197.78</pre><strong>Sample output:</strong><br /> <div style="float: none; margin: 10px 0 10px 0; text-align: center;"> <ins class="adsbygoogle" data-ad-client="ca-pub-6701402139964678" data-ad-format="auto" data-ad-slot="8447266942" data-adsbygoogle-status="done" style="display: block; height: 60px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 60px; margin: 0; padding: 0; position: relative; visibility: visible; width: 723px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 60px; margin: 0; padding: 0; position: relative; visibility: visible; width: 723px;"></ins></ins></ins> </div><pre>GeoIP Country Edition: US, United States</pre>As you see in the above output, it displays only the country location. Geoiplookup can even display more details such as the state, city, zip code, latitude and longitude etc. To do so, you need to download the city databases from Maxmind like below. Make sure you’re downloading it in /user/share/GeoIP/ location.<br /> <pre>wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz</pre><pre>gunzip GeoLiteCity.dat.gz</pre>Now, run the below command to get more details of an IP address’s geolocation.<br /> <pre>geoiplookup -f /usr/share/GeoIP/GeoLiteCity.dat 216.58.197.78</pre>Sample output would be:<br /> <pre>GeoIP City Edition, Rev 1: US, CA, California, Mountain View, 94043, 37.419201, -122.057404, 807, 650</pre>If you have saved the database files in a custom location other than the default location, you can use <strong>‘-d’</strong> parameter to specify the path. Say for example, if you have saved the database files in <strong>/home/sk/geoip/</strong>, the command to find the geolocation of an IP address would be:<br /> <pre>geoiplookup -d /home/sk/geoip/ 216.58.197.78</pre>For more details, see man pages.<br /> <pre>man geoiplookup</pre>Hope this helps. if you find this guide useful, please share it on your social networks and support us.<br /> Cheers!</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-11936751471112044202017-03-23T10:30:00.002+02:002017-03-23T10:30:53.947+02:00rtop – A Nifty Tool to Monitor Remote Server Over SSH<div dir="ltr" style="text-align: left;" trbidi="on">http://www.2daygeek.com/2017/03/rtop-monitor-remote-linux-server-over-ssh<br /><br /><span class="published"></span> <br /> <div class="entry themeform"><div class="entry-inner"> <div style="float: left; margin: 10px 10px 10px 0;"> <ins class="adsbygoogle" data-ad-client="ca-pub-7730570141079407" data-ad-slot="8826511175" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins> </div><a href="http://www.rtop-monitor.org/" target="_blank">rtop</a> is a simple, agent-less, remote server monitoring tool that works over SSH. It doesn’t required any other software to be installed on remote machine, except openSSH server package &amp; remote server credentials.<br /> rtop is written in <strong><a href="http://www.2daygeek.com/install-go-language-linux/" target="_blank">golang</a></strong>, and requires Go version 1.2 or higher. It can able to monitor any modern Linux distributions. rtop can connect remote system with all possible way like using ssh-agent, private keys or password authentication. Choose the desired one and monitor it.<br /> It works by establishing an SSH session, and running commands on the remote server to collect system metrics such as CPU, disk, memory, network. It keeps refreshing the information every few seconds, like <strong><a href="http://www.2daygeek.com/top-command-examples-to-monitor-server-performance/" target="_blank">top command</a></strong> utility.<br /> <div style="background-color: #dbe7f8; border: 1px solid #b0cbf0; color: #1e529c; padding: 15px;"><strong>Suggested Read : </strong> <a href="http://www.2daygeek.com/install-go-language-linux/" style="color: #1e529c; text-decoration: none;" target="_blank">How to Install Go (Go Programming Language) in Linux</a></div><div style="background-color: #f2f9e5; border: 1px solid #ddefbc; color: #75a025; padding: 15px;"><strong>Suggested Read : </strong> <a href="http://www.2daygeek.com/category/monitoring-tools/" style="color: #75a025; text-decoration: none;" target="_blank">Linux System Monitoring Tools</a></div><h4>How to Install rtop in Linux</h4>Run <code>go get</code> command to build it. The rtop binary automatically saved under <code>$GOPATH/bin</code> and no run time dependencies or configuration needed.<br /> <pre>$ go get github.com/rapidloop/rtop<br /></pre>The rtop binary automatically saved under <code>$GOPATH/bin</code><br /> <pre>$ $GOBIN/<br />hello rtop<br />or<br />$ ls -lh /home/magi/go_proj/bin<br />total 5.9M<br />-rwxr-xr-x 1 magi magi 1.5M Mar 7 14:45 hello<br />-rwxr-xr-x 1 magi magi 4.4M Mar 21 13:33 rtop<br /></pre><h4>How to Use rtop</h4>rtop binary was present in <code>$GOPATH/bin</code>, so just run <code>$GOBIN/rtop</code> to get the usage information.<br /> <div style="float: none; margin: 10px 0 10px 0; text-align: center;"> <ins class="adsbygoogle" data-ad-client="ca-pub-7730570141079407" data-ad-format="auto" data-ad-slot="6977341179" data-adsbygoogle-status="done" style="display: block; height: 90px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 823px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 823px;"></ins></ins></ins> </div><pre>$ $GOBIN/rtop<br />rtop 1.0 - (c) 2015 RapidLoop - MIT Licensed - http://rtop-monitor.org<br />rtop monitors server statistics over an ssh connection<br /><br />Usage: rtop [-i private-key-file] [user@]host[:port] [interval]<br /><br /> -i private-key-file<br /> PEM-encoded private key file to use (default: ~/.ssh/id_rsa if present)<br /> [user@]host[:port]<br /> the SSH server to connect to, with optional username and port<br /> interval<br /> refresh interval in seconds (default: 5)<br /></pre>Just add remote host information followed by rtop command to monitor. Default refresh interval in seconds (default: 5)<br /> <pre>$ $GOBIN/rtop magi@10.30.0.1<br />magi@10.30.0.1's password: <br /><br />2daygeek.vps up 21d 16h 59m 46s<br /><br />Load:<br /> 0.13 0.03 0.01<br /><br />CPU:<br /> 0.00% user, 0.00% sys, 0.00% nice, 0.00% idle, 0.00% iowait, 0.00% hardirq, 0.00% softirq, 0.00% guest<br /><br />Processes:<br /> 1 running of 29 total<br /><br />Memory:<br /> free = 927.66 MiB<br /> used = 55.77 MiB<br /> buffers = 0 bytes<br /> cached = 40.57 MiB<br /> swap = 128.00 MiB free of 128.00 MiB<br /><br />Filesystems:<br /> /: 9.40 GiB free of 10.20 GiB<br /><br />Network Interfaces:<br /> lo - 127.0.0.1/8, ::1/128<br /> rx = 14.18 MiB, tx = 14.18 MiB<br /><br /> venet0 - 10.30.0.1/24, 2607:5300:100:200::81a/56<br /> rx = 98.76 MiB, tx = 129.90 MiB<br /></pre><div style="float: none; margin: 10px 0 10px 0; text-align: center;"> </div>Add the refresh interval manually for better monitoring. I have added 10 seconds refresh interval instead of default one (default: 5).<br /> <pre>$ $GOBIN/rtop magi@10.30.0.1 10<br />magi@10.30.0.1's password:<br /><br />2daygeek.vps up 21d 17h 7m 1s<br /><br />Load:<br /> 0.00 0.00 0.00<br /><br />CPU:<br /> 0.00% user, 0.00% sys, 0.00% nice, 0.00% idle, 0.00% iowait, 0.00% hardirq, 0.00% softirq, 0.00% guest<br /><br />Processes:<br /> 1 running of 28 total<br /><br />Memory:<br /> free = 926.83 MiB<br /> used = 56.51 MiB<br /> buffers = 0 bytes<br /> cached = 40.66 MiB<br /> swap = 128.00 MiB free of 128.00 MiB<br /><br />Filesystems:<br /> /: 9.40 GiB free of 10.20 GiB<br /><br />Network Interfaces:<br /> lo - 127.0.0.1/8, ::1/128<br /> rx = 14.18 MiB, tx = 14.18 MiB<br /><br /> venet0 - 10.30.0.1/24, 2607:5300:100:200::81a/56<br /> rx = 98.94 MiB, tx = 130.33 MiB<br /></pre><div style="float: none; margin: 10px 0 10px 0; text-align: center;"> </div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-49640879832549214292017-03-08T01:14:00.003+02:002017-03-08T01:14:59.842+02:00Linux Disable USB Devices (Disable loading of USB Storage Driver)<div dir="ltr" style="text-align: left;" trbidi="on">https://www.cyberciti.biz/faq/linux-disable-modprobe-loading-of-usb-storage-driver<br /><br /><span class="drop_cap">I</span>n our research lab, would like to disable all USB devices connected to our HP Red Hat Linux based workstations. I would like to disable USB flash or hard drives, which users can use with physical access to a system to quickly copy sensitive data from it. How do I disable USB device support under CentOS Linux, RHEL version 5.x/6.x/7.x and Fedora latest version?<br /> <span id="more-3487"></span><br /> The USB storage drive automatically detects USB flash or hard drives. You can quickly force and disable USB storage devices under any Linux distribution. The modprobe program used for automatic kernel module loading. It can be configured not load the USB storage driver upon demand. This will prevent the modprobe program from loading the usb-storage module, but will not prevent root (or another privileged program) from using the insmod/modprobe program to load the module manually. USB sticks containing harmful malware may be used to steal your personal data. It is <a href="http://www.bbc.com/news/technology-37431335" target="_blank">not uncommon</a> for USB sticks to be used to carry and transmit destructive malware and viruses to computers. The attacker can target MS-Windows, macOS (OS X), Android and Linux based system.<br /><center> <ins class="adsbygoogle" data-ad-client="ca-pub-7825705102693166" data-ad-slot="8594278667" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins> </center><h2>usb-storage driver</h2>The <kbd>usb-storage.ko</kbd> is the USB Mass Storage driver for Linux operating system. You can see the file typing the following command:<br /> <code># <strong>ls -l /lib/modules/$(uname -r)/kernel/drivers/usb/storage/usb-storage.ko</strong></code><br /> All you have to do is disable or remove the <kbd>usb-storage.ko</kbd> driver to restrict to use USB devices on Linux such as:<br /><ol><li>USB keyboards</li><li>USB mice</li><li>USB pen drive</li><li>USB hard disk</li><li>Other USB block storage</li></ol><h2>How to forbid to use USB-storage devices on using <span style="color: #009900;">fake install method</span></h2>Type the following command under <strong>CentOS or RHEL 5.x or older</strong>:<br /> <code># echo 'install usb-storage : ' &gt;&gt; /etc/modprobe.conf </code><br /> Please note that you can use <kbd>: a shell builtin or <kbd>/bin/true</kbd>.<br /> Type the following command under <strong>CentOS or RHEL 6.x/7.x or newer</strong> (including the <strong>latest version of Fedora</strong>):<br /> <code># echo 'install usb-storage /bin/true' &gt;&gt; disable-usb-storage.conf</code><br /> Save and close the file. Now the driver will not load. You can also remove USB Storage driver without rebooting the system, enter:<br /> <code># <strong>modprobe -r usb-storage</strong><br /> # <strong>mv -v /lib/modules/$(uname -r)/kernel/drivers/usb/storage/usb-storage.ko /root/</strong><br /> ##################<br /> #### verify it ###<br /> ##################<br /> # <strong>modinfo usb-storage</strong><br /> # <strong>lsmod | grep -i usb-storage</strong><br /> # lsscsi -H</code><br /> Sample outputs:</kbd><br /><div class="wp-caption aligncenter" id="attachment_145660" style="width: 609px;"><a href="https://www.cyberciti.biz/faq/linux-disable-modprobe-loading-of-usb-storage-driver/howto-disable-usb-storage-devices/" rel="attachment wp-att-145660"><img alt="Fig.01: How to disable USB mass storage devices on physical Linux system?" class="size-full wp-image-145660" height="202" src="https://s0.cyberciti.org/uploads/faq/2009/03/Howto-disable-USB-storage-devices.jpg" width="599" /></a><div class="wp-caption-text">Fig.01: How to disable USB mass storage devices on physical Linux system?</div></div><h3>Blacklist usb-storage</h3>Edit /etc/modprobe.d/blacklist.conf, enter:<br /> <code># vi /etc/modprobe.d/blacklist.conf</code><br /> Edit or append as follows:<br /><pre>blacklist usb-storage</pre>Save and close the file.<br /><h2>BIOS option</h2>You can also disable USB from system BIOS configuration option. Make sure BIOS is password protected. This is recommended option so that nobody can boot it from USB.<br /><h2>Encrypt hard disk</h2>Linux supports the various cryptographic techniques to protect a hard disk, directory, and partition. See "<a href="https://www.cyberciti.biz/hardware/howto-linux-hard-disk-encryption-with-luks-cryptsetup-command/">Linux Hard Disk Encryption With LUKS [ cryptsetup Command ]</a>" for more info.<br /><h2>Grub option</h2>You can get rid of all USB devices by disabling kernel support for USB via GRUB. Open grub.conf or menu.lst and append "nousb" to the kernel line as follows (taken from RHEL 5.x):<br /><pre>kernel /vmlinuz-2.6.18-128.1.1.el5 ro root=LABEL=/ console=tty0 console=ttyS1,19200n8 nousb</pre>Make sure you remove any other reference to usb-storage in the grub or grub2 config files. Save and close the file. Once done just reboot the system:<br /> <code># reboot</code><br /> For grub2 use /etc/default/grub config file under Fedora / Debian / Ubuntu / RHEL / CentOS Linux. I strongly suggest that you read <a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/sec-Making_Persistent_Changes_to_a_GRUB_2_Menu_Using_the_grubby_Tool.html" target="_blank">RHEL/CentOS grub2 config</a> and <a href="https://help.ubuntu.com/community/Grub2" target="_blank">Ubuntu/Debian grub2 config</a> help pages.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-74497280877482842612017-03-08T01:09:00.000+02:002017-03-08T01:09:00.521+02:00A Linux user's guide to Logical Volume Management<div dir="ltr" style="text-align: left;" trbidi="on">https://opensource.com/business/16/9/linux-users-guide-lvm<br /><br /> <br /><div class="panel-pane pane-entity-field pane-node-field-lead-image"> <div class="field field-name-field-lead-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even" rel="schema:contentUrl"><img alt="Logical Volume Management (LVM)" class="image-full-size" height="343" src="https://opensource.com/sites/default/files/styles/image-full-size/public/images/business/rh_003499_01_other11x_cc.png?itok=yxZcGx6p" title="Logical Volume Management (LVM)" width="610" /></div></div></div></div><div class="panel-pane pane-entity-field pane-file-field-file-image-caption"> <div class="field field-name-field-file-image-caption field-type-text-long field-label-inline clearfix"><div class="field-label">Image by :&nbsp;</div><div class="field-items"><div class="field-item even">opensource.com<br /> </div></div></div></div><div class="panel-pane pane-entity-field pane-node-body"> <div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even">Managing disk space has always been a significant task for sysadmins. Running out of disk space used to be the start of a long and complex series of tasks to increase the space available to a disk partition. It also required taking the system off-line. This usually involved installing a new hard drive, booting to recovery or single-user mode, creating a partition and a filesystem on the new hard drive, using temporary mount points to move the data from the too-small filesystem to the new, larger one, changing the content of the /etc/fstab file to reflect the correct device name for the new partition, and rebooting to remount the new filesystem on the correct mount point.<br /> <div class="embedded-callout-menu callout-float-right"><div class="view view-related-content-callout view-id-related_content_callout view-display-id-article_block view-dom-id-1d539b67f96cb97d3f4fe1386968e54c"> <div class="view-header"> More Linux resources<br /> </div><div class="view-content"> <div class="views-row views-row-1 views-row-odd views-row-first views-row-last"> <div class="views-field views-field-field-related-content-link"> <div class="field-content"><div class="item-list"><ul><li class="first"><a href="https://opensource.com/resources/what-is-linux?src=linux_resource_menu">What is Linux?</a></li><li><a href="https://opensource.com/resources/what-are-linux-containers?src=linux_resource_menu">What are Linux containers?</a></li><li><a href="https://opensource.com/article/16/11/managing-devices-linux?src=linux_resource_menu">Managing devices in Linux</a></li><li><a href="https://developers.redhat.com/promotions/linux-cheatsheet/?intcmp=7016000000127cYAAQ">Download Now: Linux commands cheat sheet</a></li><li class="last"><a href="https://opensource.com/tags/linux?src=linux_resource_menu">Our latest Linux articles</a></li></ul></div></div></div></div></div></div></div>I have to tell you that, when LVM (Logical Volume Manager)&nbsp;first made its appearance in Fedora Linux, I resisted it rather strongly. My initial reaction was that I did not need this additional layer of abstraction between me and the hard drives. It turns out that I was wrong, and that logical volume management is very useful.<br /> LVM&nbsp;allows for very flexible disk space management. It provides features like the ability to add disk space to a logical volume and its filesystem while that filesystem is mounted and active and it allows for the collection of multiple physical hard drives and partitions into a single volume group which can then be divided into logical volumes.<br /> The volume manager also allows reducing the amount of disk space allocated to a logical volume, but there are a couple requirements. First, the volume must be unmounted. Second, the filesystem itself must be reduced in size before the volume on which it resides can be reduced.<br /> It is important to note that the filesystem itself must allow resizing for this feature to work. The EXT2, 3, and 4 filesystems all allow both offline (unmounted) and online (mounted) resizing when increasing the size of a filesystem, and offline resizing when reducing the size. You should check the details of the filesystems you intend to use in order to verify whether they can be resized at all and especially whether they can be resized while online.<br /> <h2>Expanding a filesystem on the fly</h2>I always like to run new distributions in a VirtualBox virtual machine for a few days or weeks to ensure that I will not run into any devastating problems when I start installing it on my production machines. One morning a couple years ago I started installing a newly released version of Fedora in a virtual machine on my primary workstation. I thought that I had enough disk space allocated to the host filesystem in which the VM was being installed. I did not. About a third of the way through the installation I ran out of space on that filesystem. Fortunately, VirtualBox detected the out-of-space condition and paused the virtual machine, and even displayed an error message indicating the exact cause of the problem.<br /> Note that this problem was not due to the fact that the virtual disk was too small, it was rather the logical volume on the host computer that was running out of space so that the virtual disk belonging to the virtual machine did not have enough space to expand on the host's logical volume.<br /> Since most modern distributions use Logical Volume Management by default, and I had some free space available on the volume group, I was able to assign additional disk space to the appropriate logical volume and then expand filesystem of the host on the fly. This means that I did not have to reformat the entire hard drive and reinstall the operating system or even reboot. I simply assigned some of the available space to the appropriate logical volume and resized the filesystem—all while the filesystem was on-line and the running program, The virtual machine was still using the host filesystem. After resizing the logical volume and the filesystem I resumed running the virtual machine and the installation continued as if no problems had occurred.<br /> Although this type of problem may never have happened to you, running out of disk space while a critical program is running has happened to many people. And while many programs, especially Windows programs, are not as well written and resilient as VirtualBox, Linux Logical Volume Management made it possible to recover without losing any data and without having to restart the time-consuming installation.<br /> <h2>LVM Structure</h2>The structure of a Logical Volume Manager disk environment is illustrated by Figure 1, below. Logical Volume Management enables the combining of multiple individual hard drives and/or disk partitions into a single volume group (VG). That volume group can then be subdivided into logical volumes (LV) or used as a single large volume. Regular file systems, such as EXT3 or EXT4, can then be created on a logical volume.<br /> In Figure 1, two complete physical hard drives and one partition from a third hard drive have been combined into a single volume group. Two logical volumes have been created from the space in the volume group, and a filesystem, such as an EXT3 or EXT4 filesystem has been created on each of the two logical volumes.<br /> <div class="rtecenter"><img alt="lvm.png" height="222" src="https://opensource.com/sites/default/files/resize/images/life-uploads/lvm-520x222.png" style="height: 222px; width: 520px;" width="520" /></div><div class="rtecenter"><em><sup>Figure 1: LVM allows combining partitions and entire hard drives into Volume Groups.</sup></em></div>Adding disk space to a host is fairly straightforward but, in my experience, is done relatively infrequently. The basic steps needed are listed below. You can either create an entirely new volume group or you can add the new space to an existing volume group and either expand an existing logical volume or create a new one.<br /> <h2>Adding a new logical volume</h2>There are times when it is necessary to add a new logical volume to a host. For example, after noticing that the directory containing virtual disks for my VirtualBox virtual machines was filling up the /home filesystem, I decided to create a new logical volume in which to store the virtual machine data, including the virtual disks. This would free up a great deal of space in my /home filesystem and also allow me to manage the disk space for the VMs independently.<br /> The basic steps for adding a new logical volume are as follows.<br /> <ol><li>If necessary, install a new hard drive.</li><li>Optional: Create a partition on the hard drive.</li><li>Create a physical volume (PV) of the complete hard drive or a partition on the hard drive.</li><li>Assign the new physical volume to an existing volume group (VG) or create a new volume group.</li><li>Create a new logical volumes (LV) from the space in the volume group.</li><li>Create a filesystem on the new logical volume.</li><li>Add appropriate entries to /etc/fstab for mounting the filesystem.</li><li>Mount the filesystem.</li></ol>Now for the details. The following sequence is taken from an example I used as a lab project when teaching about Linux filesystems.<br /> <h3>Example</h3>This example shows how to use the CLI to extend an existing volume group to add more space to it, create a new logical volume in that space, and create a filesystem on the logical volume. This procedure can be performed on a running, mounted filesystem.<br /> WARNING: Only the EXT3 and EXT4 filesystems can be resized on the fly on a running, mounted filesystem. Many other filesystems including BTRFS and ZFS cannot be resized.<br /> <h3>Install hard drive</h3>If there is not enough space in the volume group on the existing hard drive(s) in the system to add the desired amount of space it may be necessary to add a new hard drive and create the space to add to the Logical Volume. First, install the physical hard drive, and then perform the following steps.<br /> <h3>Create Physical Volume from hard drive</h3>It is first necessary to create a new Physical Volume (PV). Use the command below, which assumes that the new hard drive is assigned as /dev/hdd.<br /> <pre>pvcreate /dev/hdd<br /></pre>It is not necessary to create a partition of any kind on the new hard drive. This creation of the Physical Volume which will be recognized by the Logical Volume Manager can be performed on a newly installed raw disk or on a Linux partition of type 83. If you are going to use the entire hard drive, creating a partition first does not offer any particular advantages and uses disk space for metadata that could otherwise be used as part of the PV.<br /> <h3>Extend the existing Volume Group</h3>In this example we will extend an existing volume group rather than creating a new one; you can choose to do it either way. After the Physical Volume has been created, extend the existing Volume Group (VG) to include the space on the new PV. In this example the existing Volume Group is named MyVG01.<br /> <pre>vgextend /dev/MyVG01 /dev/hdd<br /></pre><h3>Create the Logical Volume</h3>First create the Logical Volume (LV) from existing free space within the Volume Group. The command below creates a LV with a size of 50GB. The Volume Group name is MyVG01 and the Logical Volume Name is Stuff.<br /> <pre>lvcreate -L +50G --name Stuff MyVG01<br /></pre><h3>Create the filesystem</h3>Creating the Logical Volume does not create the filesystem. That task must be performed separately. The command below creates an EXT4 filesystem that fits the newly created Logical Volume.<br /> <pre>mkfs -t ext4 /dev/MyVG01/Stuff<br /></pre><h3>Add a filesystem label</h3>Adding a filesystem label makes it easy to identify the filesystem later in case of a crash or other disk related problems.<br /> <pre>e2label /dev/MyVG01/Stuff Stuff<br /></pre><h3>Mount the filesystem</h3>At this point you can create a mount point, add an appropriate entry to the /etc/fstab file, and mount the filesystem.<br /> You should also check to verify the volume has been created correctly. You can use the <b>df</b>, <b>lvs,</b> and <b>vgs</b> commands to do this.<br /> <h2>Resizing a logical volume in an LVM filesystem</h2>The need to resize a filesystem has been around since the beginning of the first versions of Unix and has not gone away with Linux. It has gotten easier, however, with Logical Volume Management.<br /> <ol><li>If necessary, install a new hard drive.</li><li>Optional: Create a partition on the hard drive.</li><li>Create a physical volume (PV) of the complete hard drive or a partition on the hard drive.</li><li>Assign the new physical volume to an existing volume group (VG) or create a new volume group.</li><li>Create one or more logical volumes (LV) from the space in the volume group, or expand an existing logical volume with some or all of the new space in the volume group.</li><li>If you created a new logical volume, create a filesystem on it. If adding space to an existing logical volume, use the resize2fs command to enlarge the filesystem to fill the space in the logical volume.</li><li>Add appropriate entries to /etc/fstab for mounting the filesystem.</li><li>Mount the filesystem.</li></ol><h3>Example</h3>This example describes how to resize an existing Logical Volume in an LVM environment using the CLI. It adds about 50GB of space to the /Stuff filesystem. This procedure can be used on a mounted, live filesystem only with the Linux 2.6 Kernel (and higher) and EXT3 and EXT4 filesystems. I do not recommend that you do so on any critical system, but it can be done and I have done so many times; even on the root (/) filesystem. Use your judgment.<br /> WARNING: Only the EXT3 and EXT4 filesystems can be resized on the fly on a running, mounted filesystem. Many other filesystems including BTRFS and ZFS cannot be resized.<br /> <h3>Install the hard drive</h3>If there is not enough space on the existing hard drive(s) in the system to add the desired amount of space it may be necessary to add a new hard drive and create the space to add to the Logical Volume. First, install the physical hard drive and then perform the following steps.<br /> <h3>Create a Physical Volume from the hard drive</h3>It is first necessary to create a new Physical Volume (PV). Use the command below, which assumes that the new hard drive is assigned as /dev/hdd.<br /> <pre>pvcreate /dev/hdd<br /></pre>It is not necessary to create a partition of any kind on the new hard drive. This creation of the Physical Volume which will be recognized by the Logical Volume Manager can be performed on a newly installed raw disk or on a Linux partition of type 83. If you are going to use the entire hard drive, creating a partition first does not offer any particular advantages and uses disk space for metadata that could otherwise be used as part of the PV.<br /> <h3>Add PV to existing Volume Group</h3>For this example, we will use the new PV to extend an existing Volume Group. After the Physical Volume has been created, extend the existing Volume Group (VG) to include the space on the new PV. In this example, the existing Volume Group is named MyVG01.<br /> <pre>vgextend /dev/MyVG01 /dev/hdd<br /></pre><h3>Extend the Logical Volume</h3>Extend the Logical Volume (LV) from existing free space within the Volume Group. The command below expands the LV by 50GB. The Volume Group name is MyVG01 and the Logical Volume Name is Stuff.<br /> <pre>lvextend -L +50G /dev/MyVG01/Stuff<br /></pre><h3>Expand the filesystem</h3>Extending the Logical Volume will also expand the filesystem if you use the -r option. If you do not use the -r option, that task must be performed separately. The command below resizes the filesystem to fit the newly resized Logical Volume.<br /> <pre>resize2fs /dev/MyVG01/Stuff<br /></pre>You should check to verify the resizing has been performed correctly. You can use the <b>df</b>, <b>lvs,</b> and <b>vgs</b> commands to do this.<br /> <h2>Tips</h2>Over the years I have learned a few things that can make logical volume management even easier than it already is. Hopefully these tips can prove of some value to you.<br /> <ul><li>Use the Extended file systems unless you have a clear reason to use another filesystem. Not all filesystems support resizing but EXT2, 3, and 4 do. The EXT filesystems are also very fast and efficient. In any event, they can be tuned by a knowledgeable sysadmin to meet the needs of most environments if the defaults tuning parameters do not.</li><li>Use meaningful volume and volume group names.</li><li>Use EXT filesystem labels.</li></ul>I know that, like me, many sysadmins have resisted the change to Logical Volume Management. I hope that this article will encourage you to at least try LVM. I am really glad that I did; my disk management tasks are much easier since I made the switch.<br /> </div></div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-87044956929292965072017-03-08T00:29:00.001+02:002017-03-08T00:29:07.164+02:00How to Create Virtual Machines in oVirt 4.0 Environment<div dir="ltr" style="text-align: left;" trbidi="on">http://www.linuxtechi.com/create-virtual-machines-ovirt-4-environment<br /><br />To Create Virtual Machines from oVirt Engine Web Administrator portal first we have to make sure following things are set.<br /> <ul><li>Data Center</li><li>Clusters</li><li>Hosts ( oVirt Node or hypervisor)</li><li>&nbsp;Network ( default ovirtmgmt is created)</li><li>Storage Domain( ISO storage and Data Storage )</li></ul>In our previous article we have already discuss the&nbsp; oVirt Engine and oVirt Node / Hypervisor installation. Please refer the URL for “<span style="color: navy;"><a href="http://www.linuxtechi.com/install-configure-ovirt-4-0-on-centos7-rhel7/" style="color: navy;" target="_blank">Installation Steps of oVirt Engine and Ovirt Node</a></span>”<br /> Refer the following steps to complete above set of tasks. Login to your oVirt Engine Web Administrator Portal. In my Case web portal URL is “https://ovirtengine.example.com”<br /> <h4><span style="color: purple;">Step:1 Create new Data Center</span></h4>Go to Data Centers Tab and then click on New<br /> Specify the Data Center Name, Description and Storage Type and Compatibility version.In my case Data Center name is “<strong>test_dc</strong>”<br /> <a data-rel="lightbox-0" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Create-Data-Center-from-oVirt-Engine-Web-Administrator-Portal.jpg" title=""><img alt="create-data-center-from-ovirt-engine-web-administrator-portal" class="wp-image-4546 aligncenter" height="424" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Create-Data-Center-from-oVirt-Engine-Web-Administrator-Portal-1024x557.jpg" width="779" /></a><br /> <h4><span style="color: purple;">Step:2 Configure Cluster for Data Center</span></h4>When we click on OK on above step, it will ask to configure Cluster. So Select “<strong>Configure Cluster</strong>” option<br /> <a data-rel="lightbox-1" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Configure-Cluster-from-oVirt-Engine-Web-portal.jpg" title=""><img alt="configure-cluster-from-ovirt-engine-web-portal" class="wp-image-4544 aligncenter" height="427" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Configure-Cluster-from-oVirt-Engine-Web-portal-1024x557.jpg" width="785" /></a><br /> Specify the cluster name, Description, CPU architecture as per your setup leave the other parameters as it is. We can define optimization, migration and fencing policy as per our requirement&nbsp; but i am not touching these policy as of now.<br /> In my case Cluster name is “<strong>testcluster</strong>”<br /> <a data-rel="lightbox-2" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/New-Cluster-oVirt-Engine-Web-Administration.jpg" title=""><img alt="new-cluster-ovirt-engine-web-administration" class="wp-image-4556 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/New-Cluster-oVirt-Engine-Web-Administration-1024x557.jpg" width="786" /></a><br /> Click on OK.<br /> In the next step click on Configure Later.<br /> <a data-rel="lightbox-3" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Configure-Host-Later-oVirt-Engine-Web-Administration.jpg" title=""><img alt="configure-host-later-ovirt-engine-web-administration" class="wp-image-4545 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Configure-Host-Later-oVirt-Engine-Web-Administration-1024x557.jpg" width="786" /></a><br /> <h4><span style="color: purple;">Step:3 Add Host or oVirt Node to above created data center &amp; cluster.</span></h4>By default when we add any host or oVirt Node in oVirt Engine it is added to the default data center and Cluster. So to change the data center and cluster of any node first put the host in maintenance mode<br /> Select the Node click on Maintenance option then click on OK<br /> <a data-rel="lightbox-4" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Maintenance-Mode-Host-Ovirt-Engine.jpg" title=""><img alt="maintenance-mode-host-ovirt-engine" class="wp-image-4555 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Maintenance-Mode-Host-Ovirt-Engine-1024x557.jpg" width="786" /></a><br /> Now Select the Edit option and update the Data center and Cluster information for the selected host.<br /> <a data-rel="lightbox-5" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Update-DataCenter-Cluster-info-ovirt-engine.jpg" title=""><img alt="update-datacenter-cluster-info-ovirt-engine" class="wp-image-4558 aligncenter" height="429" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Update-DataCenter-Cluster-info-ovirt-engine-1024x557.jpg" width="788" /></a><br /> Click on OK<br /> Now Click on Activate option to activate the host.<br /> <a data-rel="lightbox-6" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Host-Activated-ovirt-engine-1.jpg" title=""><img alt="host-activated-ovirt-engine" class="wp-image-4551 aligncenter" height="340" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Host-Activated-ovirt-engine-1.jpg" width="788" /></a><br /> <ins class="adsbygoogle" data-ad-client="ca-pub-2130177517648108" data-ad-format="auto" data-ad-slot="3458852898" style="display: block;"></ins> <h4><span style="color: purple;">Step:4 Creating Storage Domains</span></h4>As the name suggests storage domain is centralized repository of disk which is used for storing the VM disk images, ISO files and VMs meta data its Snapshots. Storage Domain is classified into three types :<br /> <ul><li><strong>Data Storage Domain</strong> : It is used for storing hard disk images of all the VMs</li><li><strong>Export Storage Domain</strong> : It is used to store the backup copies of VMs, it also provides transitory storage for hard disk images and templates being transferred between data centers.</li><li><strong>ISO Storage Domain</strong> : It is used for storing the ISO files.</li></ul>In this article Data Storage and ISO storage is shared via NFS. Though data storage can be configure via ISCSI , GlusterFS and Storage using Fibre Channels. Following NFS share is available for Data Storage and ISO domain.<br /> <pre>[root@ovirtnode ~]# showmount -e 192.168.1.30<br />Export list for 192.168.1.30:<br />/exports/vmstorage 192.168.1.0/24<br />/exports/iso&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 192.168.1.0/24<br />[root@ovirtnode ~]#</pre>Create Data Storage Domain, Click on the Storage Tab and then Click on New Domain, Select the <strong>Domain function</strong> as “Data” and <strong>Storage Type</strong> as NFS and Specify the NFS servers’ share ip and name.<br /> <a data-rel="lightbox-7" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Data-Storage-Domain-ovirt-engine.jpg" title=""><img alt="data-storage-domain-ovirt-engine" class="wp-image-4548 aligncenter" height="429" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Data-Storage-Domain-ovirt-engine-1024x557.jpg" width="788" /></a><br /> Now Again Click on New Domain from Storage Tab and Select Domain Function as “ISO” and Storage Type as “NFS”<br /> <a data-rel="lightbox-8" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/ISO-Storage-Domain-oVirt-Engine.jpg" title=""><img alt="iso-storage-domain-ovirt-engine" class="wp-image-4553 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/ISO-Storage-Domain-oVirt-Engine-1024x557.jpg" width="786" /></a><br /> As we see both the storage Domains are activated now. Once the storage Domain got activated then automatically our Data Center initialized and becomes active.<br /> <a data-rel="lightbox-9" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Storage-Domain-Activated-ovirt-engine.jpg" title=""><img alt="storage-domain-activated-ovirt-engine" class="wp-image-4557 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Storage-Domain-Activated-ovirt-engine-1024x557.jpg" width="786" /></a><br /> <h4><span style="color: purple;">Step:5 Upload ISO files to ISO Storage Domain.</span></h4>Transfer the ISO file to ovirt-engine and run the ‘<strong>engine-iso-uploader</strong>’. In my case i am uploading Ubuntu 16.04 LTS iso file.<br /> <pre>[root@ovirtengine ~]# engine-iso-uploader -i ISO_Domain_test_dc upload ubuntu-16.04-desktop-amd64.iso<br />Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort):<br />Uploading, please wait...<br />INFO: Start uploading ubuntu-16.04-desktop-amd64.iso<br />Uploading: [########################################] 100%<br />INFO: ubuntu-16.04-desktop-amd64.iso uploaded successfully<br />[root@ovirtengine ~]#</pre>Now we are ready create Virtual machines.<br /> <h4><span style="color: purple;">Step:6 Create Virtual Machine</span></h4>As we have uploaded Ubuntu 16.04 ISO file so at this point of time we will create Ubuntu virtual machine.<br /> Click on New VM from Virtual Machine Tab . Specify the followings parameters under the “<strong>General</strong>” Tab<br /> <ul><li>Data Center “test_dc”</li><li>Operating System Type as “Linux”</li><li>Optimized for “Desktop”</li><li>Name as “Ubuntu 16.04”</li><li>&nbsp;nic1 as “ovirtmgmt”</li></ul><a data-rel="lightbox-10" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/create-new-virtual-machine-ovirt-engine.jpg" title=""><img alt="create-new-virtual-machine-ovirt-engine" class="wp-image-4547 aligncenter" height="427" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/create-new-virtual-machine-ovirt-engine-1024x557.jpg" width="785" /></a><br /> Specify the disk space for the Virtual machine. Click on Create option&nbsp; which is available in front of “<strong>Instance Images</strong>” Specify the Disk Size and leave other parameters as it and click on OK.<br /> <a data-rel="lightbox-11" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Disk-image-for-virtual-machine-ovirt-engine.jpg" title=""><img alt="disk-image-for-virtual-machine-ovirt-engine" class="wp-image-4550 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Disk-image-for-virtual-machine-ovirt-engine-1024x557.jpg" width="786" /></a><br /> Click on “<strong>Show Advance option</strong>” then Go to System Tab, Specify the Memory and CPU for the Virtual Machine<br /> <a data-rel="lightbox-12" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/define-memory-cpu-virtual-machine-ovirt-engine.jpg" title=""><img alt="define-memory-cpu-virtual-machine-ovirt-engine" class="wp-image-4549 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/define-memory-cpu-virtual-machine-ovirt-engine-1024x557.jpg" width="786" /></a><br /> Go to “<strong>Boot Options</strong>” Tab , attach the Ubuntu 16.04 ISO file and change the boot sequence and Click on OK<br /> <a data-rel="lightbox-13" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/attach-iso-file-virtual-machine-ovirt-engine.jpg" title=""><img alt="attach-iso-file-virtual-machine-ovirt-engine" class="wp-image-4543 aligncenter" height="427" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/attach-iso-file-virtual-machine-ovirt-engine-1024x557.jpg" width="785" /></a><br /> Now Select the VM and Click on “<strong>Run Once</strong>” option from Virtual Machines Tab.<br /> To Get the Console of&nbsp; VM. Do the Right Click on VM and then select Console.<br /> <a data-rel="lightbox-14" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Install-Ubuntu-from-ovirt-engine-console.jpg" title=""><img alt="install-ubuntu-from-ovirt-engine-console" class="wp-image-4552 aligncenter" height="428" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Install-Ubuntu-from-ovirt-engine-console-1024x557.jpg" width="786" /></a><br /> Click on Install Ubuntu and follow the screen instructions and reboot the VM once installation is completed<br /> <a data-rel="lightbox-15" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Virtual-machine-Installation-completed-ovirt-engine.jpg" title=""><img alt="virtual-machine-installation-completed-ovirt-engine" class="wp-image-4559 aligncenter" height="422" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Virtual-machine-Installation-completed-ovirt-engine-1024x550.jpg" width="786" /></a><br /> Change the Boot Sequence of VM so that it will boot from Disk.<br /> <a data-rel="lightbox-16" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Login-Screen-Virtual-Machine-ovirt-engine.jpg" title=""><img alt="login-screen-virtual-machine-ovirt-engine" class="wp-image-4554 aligncenter" height="429" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Login-Screen-Virtual-Machine-ovirt-engine-1024x558.jpg" width="787" /></a><br /> Enter the Credentials that you set during installation<br /> <a data-rel="lightbox-17" href="http://www.linuxtechi.com/wp-content/uploads/2016/09/Virtual-Machine-Terminal-ovirt-engine.jpg" title=""><img alt="virtual-machine-terminal-ovirt-engine" class="wp-image-4560 aligncenter" height="420" src="http://www.linuxtechi.com/wp-content/uploads/2016/09/Virtual-Machine-Terminal-ovirt-engine-1024x549.jpg" width="784" /></a><br /> That’s all for this article. Hope you under stand how to create or deploy virtual machines in oVirt Environment.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-52273437782070568572017-03-08T00:15:00.002+02:002017-03-08T00:15:46.095+02:0010 tips for DIY IoT home automation<div dir="ltr" style="text-align: left;" trbidi="on">https://opensource.com/life/16/9/iot-home-automation-projects<br /><br /><div class="os-article__header"> <div class="panel-pane pane-node-title"> <h1></h1></div><div class="panel-pane pane-entity-field pane-node-field-lead-image"> <div class="field field-name-field-lead-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even" rel="schema:contentUrl"><img alt="10 tips for DIY IoT home automation" class="image-full-size" height="343" src="https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/LIFE_housing.png?itok=q6XaNddR" title="10 tips for DIY IoT home automation" width="610" /></div></div></div></div><div class="panel-pane pane-entity-field pane-file-field-file-image-caption"> <div class="field field-name-field-file-image-caption field-type-text-long field-label-inline clearfix"><div class="field-label">Image by :&nbsp;</div><div class="field-items"><div class="field-item even">opensource.com<br /> </div></div></div></div></div><div class="panel-pane pane-entity-field pane-node-body"> <div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even">We live in an exciting time. Everyday more <em>things</em>&nbsp;become <em>Internet-connected things</em>.&nbsp;They have&nbsp;sensors and can communicate with other things, and help us perform tasks like never before. Especially at home.<br /> Home automation is made possible by amaetuer developers and tinkers because the price of microcontrollers with the ability to talk over a network continue to drop. It all started for me when I was stuck in the office wishing I was at home playing with my kids. Since I couldn't be there physically,&nbsp;I&nbsp;built a squirt gun out of a microcontroller, a couple of servos, a solenoid valve,&nbsp;and a water hose for around $80 US. <a href="https://www.youtube.com/watch?v=FWINsKcP8oQ&amp;feature=youtu.be" target="_blank">See how I did it</a>.<br /> I was on to something. Next I&nbsp;built what I call The Logical Living&nbsp;home automation system&nbsp;out of inexpensive microcontrollers, custom circuits,&nbsp;and other household components. And, I <a href="http://www.codeproject.com/Articles/854907/IoT-for-Home-Automation" target="_blank">published the code</a> at&nbsp;Code Project.&nbsp;My house now has hundreds of IoT features helping me help it run effeciently, and with more input from me, the home owner.<br /> Along the way, I've learned a few things that can help other beginner IoT makers.<br /> <h2>6 design&nbsp;lessons for getting started</h2><strong>Lesson 1:&nbsp;Make each thing smart.</strong><br /> It is hard to move things around when all of your things are connected with wires to a central controller. If each "thing" is self-contained, then it's easy to move it around and easy to take it with you when you move.<br /> <strong>Lesson 2: Update the program (firmware) Over The Air (OTA).</strong><br /> It is important to select a microcontroller or microprocessor that has the capability to flash code updates to your remote device.&nbsp;I built a 20 foot outdoor Christmas tree made of lights that I can program while sitting in the office or anywhere with an Internet connection.&nbsp;This is especially nice when it is cold and raining outside.&nbsp;It is very inconvenient to plug my laptop into some of my other IoT projects to do code updates.&nbsp;There is a simple feature that I have been wanting to add for a long time to an IoT cat toy project built on a different platform but the pain of connecting my laptop to the hard to access microcontroller has kept me from making the update.<br /> <strong>Lesson 3:&nbsp;Use DHCP or an identity service.</strong><br /> And&nbsp;have one program for all of the devices for each type of microcontroller in your fleet.<br /> <strong>Lesson 4:&nbsp;Use a <em>publish / subscribe</em> model.</strong><br /> Do so with a broker to loosely couple all of your things.&nbsp;A broker is software middleware between the "thing" and whatever is communicating with it.&nbsp;Many of my previous IoT implementations were done with "things" that were tightly coupled to a broker to dispatch messages to other "things".&nbsp;I have learned that a well-designed broker can connect publishers with subscribers in a loose coupled approach without opening up a port in the firewall.&nbsp;It is a smart idea to leverage MQTT protocol and an open source broker like Mosquitto.<br /> <strong>Lesson 5:&nbsp;Leverage existing cloud services.</strong><br /> Machine learning algorithms can be complex and you can develop new features much quicker by leveraging work from large teams of people with specialties in the area. I'm working on an IoT project to predict the health of my pets that I would not have the time to get the expertise to do without the help from existing cloud services.<br /> <strong>Lesson 6:&nbsp;Make the code available to the community.</strong><br /> When I open sourced the code and made it available to the community, I put extra time and thought into making sure the code was clean, of high quality, and used best practices. I knew that many eyes would be looking at and reviewing the code which caused me to want to&nbsp;refactor it often. Open sourcing your project is a great way to get feedback from the community and improve.<br /> <h2>4 tips for IoT in the home&nbsp;</h2>I've learned just as many lessons about people as I did about technology.<br /> <strong>Lesson 1: With great power comes great responsibility.</strong><br /> I can control the TV, DVR, and music player with&nbsp;IR signals. So, to be funny,&nbsp;I'd randomly change&nbsp;the TV channel&nbsp;or music station&nbsp;when I was away from home, while my family was at home. It was my way of telling them&nbsp;I was thinking of them, but they didn't exactly&nbsp;see it that way! When I got home someone had disabled the control by removing wires from my circuit.&nbsp;Needless to say,&nbsp;I was proud they figured out which wires to remove to disable it. Smart!<br /> <strong>Lesson 2: Be aware of pets.</strong><br /> We have a cat that likes to play in funny places, and she was particularly interested in&nbsp;my project to&nbsp;control the fireplace with speech-voice recognition. A burned kitty would mean the end of my IoT projects, so I quickly wired up a mesh screen to keep the cat out.<br /> <strong>Lesson 3: Beware of fire.</strong><br /> I&nbsp;built an IoT-controlled pumpkin for Halloween that shot a 4-foot flame out of its face when mentioned on Twitter or alternatively controlled with a watch or phone. This was a huge hit&nbsp;but IT became difficult to keep all the kids at a&nbsp;safe distance all night long. This year, I'm building a 12-foot monster that shoots the flame way above the kids heads and is controlled by speech commands. See some of <a href="https://www.youtube.com/watch?v=H1cJL4WBQMI" target="_blank">my other Halloween IoT projects</a>.<br /> <strong>Lesson 4:&nbsp;When it's in the home it needs to be nearly 100% reliable.</strong><br /> Family members are not forgiving of quality defects, and your home automation projects will not be used if they are not reliable.<br /> Some of my microcontrollers would lock up after a couple of days because of Ethernet communication issues, and&nbsp;I knew I had a problem&nbsp;when my wife called me while I was traveling&nbsp;because the garden wasn't watering. I spent days working out the issue and finally resolved it by having the code detect the issue and then reboot the device to recover. The reboot is so fast that people don't usually notice the downtime.<br /> </div></div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-21739287744536208222017-03-08T00:07:00.001+02:002017-03-08T00:07:11.391+02:00 5 Tips on Using OAuth 2.0 for Secure Authorization <div dir="ltr" style="text-align: left;" trbidi="on">http://www.esecurityplanet.com/mobile-security/5-tips-on-using-oauth-2.0-for-secure-authorization.html<br /><br /><h3> OAuth 2.0 can be an effective authorization method. Here we offer tips on implementing and using an OAuth 2.0 authorization server using the OWIN framework.<br /> </h3><div class="byline-share"></div><div class="top-inset-left"> <img alt="null" height="165" src="http://www.esecurityplanet.com/imagesvr_ce/7707/tan-code-250x165.jpg" width="250" /> </div><em>By Aleksey Gavrilenko, Itransition</em><br /> Approaches to security issues change constantly, along with evolving threats. One approach is to implement OAuth, an open authorization standard that provides secure access to server resources. OAuth is a broad topic with hundreds of articles covering dozens of its aspects. This particular article will help you create a secure authorization server using OAuth 2.0 in .NET to use for your mobile clients and web applications.<br /> <h2>What is OAuth?</h2>OAuth is an open standard in authorization that allows delegating access to remote resources without sharing the owner's credentials. Instead of credentials, OAuth introduces tokens generated by the authorization server and accepted by the resource owner.<br /> In OAuth 1.0, each registered client was given a client secret and the token was provided in response to an authentication request signed by the client secret. That produced a secure implementation even in the case of communicating through an insecure channel, because the secret itself was only used to sign the request and was not passed across the network.<br /> <div class="in-content-imu"> </div>OAuth 2.0 is a more straightforward protocol passing the client secret with every authentication request. Therefore, this protocol is not backward compatible with OAuth 1.0. Moreover, it is deemed less secure because it relies solely on the SSL/TLS layer. One of OAuth contributors, Eran Hammer, even said that OAuth 2.0 may become "the road to hell," because:<br /> <em>"… OAuth 2.0 at the hand of a developer with deep understanding of web security will likely result in a secure implementation. However, at the hands of most developers – as has been the experience from the past two years – 2.0 is likely to produce insecure implementations."</em><em></em><br /> Despite this opinion, making a secure implementation of OAuth 2.0 is not that hard, because there are frameworks supporting it and best practices listed. SSL itself is a very reliable protocol that is impossible to compromise when proper certificate checks are thoroughly performed.<br /><div class="FIOnDemandWrapper" id="FIOnDemandWrapper_fiInstance_100583_0_4958176" style="clear: both; display: block; float: none; margin: 0px auto; padding-bottom: 35px; padding-top: 5px;"><div class="apd_static_banner " style="clip: auto; height: 250px; margin: 0px auto; width: 300px;"><div style="color: silver; padding-bottom: 5px; text-align: center;">-&nbsp;Advertisement&nbsp;-</div></div></div>Of course, if you are using OAuth 1.0, then continue to use it; there is no point in migrating to OAuth 2.0. But if you are developing a new mobile or an Angular web application (and often mobile and web applications come together, sharing the same server), then OAuth 2.0 will be a better choice. It already has some built-in support in the <a href="http://owin.org/" target="_blank">OWIN framework</a> for .NET that can be easily extended to create different clients and use different security settings.<br /> <h2>Implementing OAuth 2.0 in OWIN</h2>OWIN is a .NET framework for building ASP.NET Web API applications. It offers its own implementation of OAuth 2.0 protocol where two major OAuth terms (clients and refresh tokens) are not strictly defined and need to be implemented separately. On the one hand, it adds some complexity -- because each developer needs to decide how to implement them exactly -- and, on the other hand, it adds the extensibility and new opportunities.<br /> <img alt="OAuth1st" height="277" src="http://www.esecurityplanet.com/imagesvr_ce/8354/OAuthTop.jpg" width="595" /><br /> <img alt="Oauth2nd" height="409" src="http://www.esecurityplanet.com/imagesvr_ce/3404/OAuthbottom.jpg" width="595" /><br /> The exact implementation with code snippets can be found in tutorials across the web and in open source projects at GitHub; and therefore it is out of scope of the current article. In particular, Taiseer Joudeh, a Microsoft consultant, has written <a href="http://bitoftech.net/2014/07/16/enable-oauth-refresh-tokens-angularjs-app-using-asp-net-web-api-2-owin/" target="_blank">an article </a>with a step-by-step description of the exact implementation.<br /> From my own experience, it's best to use the following techniques when implementing and using an OAuth 2.0 authorization server:<br /> <ol><ol><ol><li><strong>Always use SSL</strong>. OAuth 2.0 security depends solely on SSL and using OAuth 2.0 without it is just like sending a password in a plaintext across an insecure Wi-Fi connection.</li><li><strong>Always check the SSL certificate</strong> to protect from the man-in-the-middle attacks. For web applications, the browser does that job and warns the user if the certificate is not to be trusted. For mobile applications, the application itself should check the certificate for validity.</li><li><strong>Do not store client secrets in the database in plaintext</strong>; store the hashed value instead. You may choose not to store client secrets at all (which is an acceptable solution if the authentication relies solely on passwords), but keeping them in plaintext will pose a security threat if they become critical in the future.</li><li><strong>Always use refresh tokens</strong> and make access tokens short-lived. Using refresh tokens will give you the following three benefits:</li><ul><li>They can be used to avoid access tokens living forever and not forcing the user to re-enter credentials at the same time. As a bonus, for web applications they can be used to imitate session expiration. When the user is idle for some time, both the access and the refresh token will expire and the user will be forced to re-login.</li><li>They are revocable. When the user changes the password, the token can be revoked and the user will be forced to re-login on all mobile devices. This is very important because a device may be stolen and having a logged-in session on it will pose a significant security threat.</li><li>They can be used for updating access token content. Normally, access tokens are validated without a roundtrip to the database. This makes it faster to process, but user roles (that are cached in claims) may not be easily updated or, even more importantly, revoked if access token expiration takes a long time. Refresh tokens are of great help here because they shorten the access tokens' life.</li></ul><li><strong>Choose the lifetime for access tokens</strong> and refresh tokens properly. For financial or other critical applications, the token's lifetime should be as short as possible: 30-60 seconds for access tokens and five to 10 minutes for refresh tokens. Non-critical applications may have refresh tokens living for weeks so that users are not bothered with re-entering credentials.</li></ol></ol></ol><h2>OWIN Implementation of OAuth 2.0 Offers Flexibility</h2>Also, current OWIN implementation of OAuth 2.0 is flexible enough to be altered to fit particular business needs:<br /> <ol><ol><ol><ol><li>If there is a background service that needs to act as any user, it can be integrated seamlessly into the authentication process in the following way:</li><ul><li>Alter the clients table by adding a PasswordRequired column.</li><li>Handle the case when the password is not required in the source code.</li><li>Create a new client in the clients table and use it for the background service. Always secure the secret for this client as it will act like the master password. (Never store this secret in plaintext.)</li></ul><li>If there are several applications (mobile apps, admin console, etc.) that need to be restricted by roles, you can protect the client applications in the following way:</li><ul><li>Alter the clients table by adding an AllowedRoles column.</li><li>Implement additional checks for the user role to the authentication code.</li><li>Dedicate different rows in the client's table for each application. Remember that the authorization checks in the server API must be implemented in any case.</li></ul><li>Sometimes the requirements may be vice versa: the same user logging in through different applications should have different business roles when accessing the server resources. In this case, the client's table can be altered by adding and maintaining a new BusinessRole column. The value from this column can be added to the access token claims to be eventually checked in the server API.</li></ol></ol></ol></ol><h2>Remember, No Authentication Method Is Perfect</h2>There is no ideal way to protect users from attacks when using applications, and even OAuth 2.0 has advantages and flaws exposed in implementations. By avoiding implementation mistakes and using the methods described in the article above, developers can help users stay more secure without breaking the seamless interaction with the app.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-24382147796919016142017-03-07T23:48:00.004+02:002017-03-07T23:48:44.457+02:00Create Virtual Machine Template in oVirt Environment<div dir="ltr" style="text-align: left;" trbidi="on">http://www.linuxtechi.com/create-vm-template-ovirt-environment<br /><br />A template is a pre-installed and pre-configured virtual machine and Templates become beneficial where we need to deploy large number similar virtual machines.Templates help us to reduce the time to deploy virtual machine and also&nbsp; reduce the amount of disk space needed.A template does not require to be cloned. Instead a small overlay can be put on top of the base image to store just the changes for one particular instance.<br /> To Convert a virtual machine into a template we need to generalize the virtual machine or in other words sealing virtual machine.<br /> In our previous articles we have already discuss the following topics.<br /> <ul><li><h6><span style="color: maroon;"><a href="http://www.linuxtechi.com/install-configure-ovirt-4-0-on-centos7-rhel7/" style="color: maroon;" target="_blank">Installation and configuration of oVirt 4.0</a></span></h6></li><li><h6><a href="http://www.linuxtechi.com/create-virtual-machines-ovirt-4-environment/" target="_blank"><span style="color: maroon;">Creating Virtual machines in oVirt Environment</span> </a></h6></li></ul>I am assuming either CentOS 7 or RHEL 7 Virtual is already deployed in oVirt environment. We will be using this virtual machine and will convert it into a template. Refer the following steps :<br /> <h4><span style="color: purple;">Step:1 Login to Virtual Machine Console</span></h4>SSH the virtual&nbsp; machine as a root user.<br /> <h4><span style="color: purple;">Step:2 Remove SSH host keys&nbsp; using rm command.</span></h4><pre>[root@linuxtechi ~]# rm -f /etc/ssh/ssh_host_*</pre><h4><span style="color: purple;">Step:3 Remove the hostname and set it as local host</span></h4><pre>[root@linuxtechi ~]# hostnamectl set-hostname 'localhost'</pre><h4><span style="color: purple;">Step:4 Remove the host specific information</span></h4>Remove the followings :<br /> <ul><li>udev rules</li><li>MAC Address &amp; UUID</li></ul><pre>[root@linuxtechi ~]# rm -f /etc/udev/rules.d/*-persistent-*.rules<br />[root@linuxtechi ~]# sed -i '/^HWADDR=/d' /etc/sysconfig/network-scripts/ifcfg-*<br />[root@linuxtechi ~]# sed -i '/^UUID=/d' /etc/sysconfig/network-scripts/ifcfg-*</pre><h4><span style="color: purple;">Step:5 Remove RHN systemid associated with virtual machine</span></h4><pre>[root@linuxtechi ~]# rm -f /etc/sysconfig/rhn/systemid</pre><h4><span style="color: purple;">Step:6 Run the command sys-unconfig</span></h4>Run the command <strong>sys-unconfig</strong> to complete the process and it will also shutdown the virtual machine.<br /> <pre>[root@linuxtechi ~]# sys-unconfig</pre><h4><span style="color: purple;">Now our Virtual Machine is ready for template.</span></h4>Do the right click on the Machine and select the “<strong>Make Template</strong>” option<br /> <a data-rel="lightbox-0" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/Create-Template-from-virtual-machine-ovirt.jpg" title=""><img alt="create-template-from-virtual-machine-ovirt" class=" wp-image-4569 aligncenter" height="384" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/Create-Template-from-virtual-machine-ovirt.jpg" width="782" /></a><br /> Specify the Name and Description of the template and click on OK<br /> <a data-rel="lightbox-1" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/template-specification-ovirt.jpg" title=""><img alt="template-specification-ovirt" class="aligncenter wp-image-4572" height="425" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/template-specification-ovirt-1024x557.jpg" width="781" /></a><br /> It will take couple of minutes to create template from the virtual machine. Once Done go to templates Tab and verify whether the newly created template is there or not.<br /> <a data-rel="lightbox-2" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/centos7-vm-template-ovirt.jpg" title=""><img alt="centos7-vm-template-ovirt" class="aligncenter wp-image-4568" height="425" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/centos7-vm-template-ovirt-1024x557.jpg" width="781" /></a><br /> <ins class="adsbygoogle" data-ad-client="ca-pub-2130177517648108" data-ad-format="auto" data-ad-slot="3458852898" style="display: block;"></ins> <h4><span style="color: purple;">Now start deploying virtual machine from template.</span></h4>Got to the Virtual Machine Tab , click on “<strong>New VM</strong>“, Select the template that we have created in above steps. Specify the VM name and Description<br /> <a data-rel="lightbox-3" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/deploy-virtual-machine-from-template-ovirt.jpg" title=""><img alt="deploy-virtual-machine-from-template-ovirt" class="aligncenter wp-image-4571" height="426" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/deploy-virtual-machine-from-template-ovirt-1024x557.jpg" width="783" /></a><br /> When we click on OK , it will start creating the virtual machine from template. Example is shown below :<br /> <a data-rel="lightbox-4" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/creating-vm-template-ovirt.jpg" title=""><img alt="creating-vm-template-ovirt" class="aligncenter wp-image-4570" height="425" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/creating-vm-template-ovirt-1024x557.jpg" width="781" /></a><br /> As we can see that after couple of minutes Virtual Machine “test_server1” has been successfully launched from template.<br /> <a data-rel="lightbox-5" href="http://www.linuxtechi.com/wp-content/uploads/2016/10/VM-successfuly-launched-from-template-ovirt.jpg" title=""><img alt="vm-successfuly-launched-from-template-ovirt" class="aligncenter wp-image-4573" height="427" src="http://www.linuxtechi.com/wp-content/uploads/2016/10/VM-successfuly-launched-from-template-ovirt-1024x559.jpg" width="783" /></a><br /> That’s all, hope you got an idea how to create a template from a Virtual machine.Please share your feedback and comments.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-7152646119007591012017-03-07T23:15:00.001+02:002017-03-07T23:15:25.941+02:00Linux Directory Structure (File System Hierarchy) Explained with Examples<div dir="ltr" style="text-align: left;" trbidi="on">http://www.2daygeek.com/linux-directory-structure-file-system-hierarchy<br /><br />Are you new to Linux ? If so, I would advise you to understand the Linux Directory Structure (File System Hierarchy) first. Don’t panic/scare after seeing the below image (File System Hierarchy). Getting confusion about <code>/bin</code>, <code>/sbin</code>, <code>/usr/bin</code> &amp; <code>/usr/sbin</code> don’t worry, we are here to teach you like a baby.<br /> The <strong>Filesystem Hierarchy Standard</strong> (FHS) defines the structure of file systems in Unix/Linux, like operating systems.<br /> In Linux everything is a file, we can modify anything whenever it’s necessary but make sure, you should know what you are doing. If you don’t know, what you are doing &amp; did something without knowing anything which will damage your system potentially. So try to learn from basic to avoid such kind of issues on production environment.<br /><a href="http://www.2daygeek.com/wp-content/uploads/2016/10/linux-file-system-structure-final-4.png"><img alt="linux-file-system-structure-final-4" class="aligncenter size-full wp-image-12510" data-lazy-loaded="true" height="543" src="http://www.2daygeek.com/wp-content/uploads/2016/10/linux-file-system-structure-final-4.png" style="display: block;" width="811" /></a><br /> <div style="float: none; margin: 10px 0 10px 0; text-align: center;"> <ins class="adsbygoogle" data-ad-client="ca-pub-7730570141079407" data-ad-format="auto" data-ad-slot="6977341179" data-adsbygoogle-status="done" style="display: block; height: 90px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 909px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 909px;"></ins></ins></ins> </div><ul><li><strong><code>/</code> : The Root Directory</strong> – Primary hierarchy root and root directory of the entire file system hierarchy which contains all other directories and files. Make a note <code>/</code> &amp; <code>/root</code> is different.</li><li><strong><code>/bin</code> : Essential User Binaries</strong> – Contains Essential User Binaries, where all the users performing most commonly used basic commands like ps, ls, ping, grep, cp &amp; cat</li><li><strong><code>/boot</code> : Static Boot Files</strong> – Contains boot loader related files which is needed to start up the system, such as Kernel initrd (Initial RAM Disk image), vmlinuz (Virtual Memory LINUx gZip – compressed Linux kernel Executable) &amp; grub (Grand Unified Bootloader). Make a note, its a <code>vmlinuz</code> not a <code>vmlinux</code> vmlinuz – Virtual Memory LINUX, Non-compressed Linux Kernel Executable</li><li><strong><code>/dev</code> : Device Files</strong> – contains all device files for various hardware devices on the system, including hard drives, RAM, CPU, tty, cdrom, etc,. It’s not a regular files.</li><li><strong><code>/etc</code> : Configuration Files</strong> – contains system global configuration files, which affect the system’s behavior for all users when you modifying anything on it. Also having application service script, like (start, stop, enable, shutdown &amp; status).</li><li><strong><code>/home</code> : User’s Home Directories</strong> – Users’ home directories, where users can save their persona files.</li><li><strong><code>/lib</code> : Essential Shared Libraries</strong> – Contains important dynamic libraries and kernel modules that supports the binaries found under /bin &amp; /sbin directories.</li><li><strong><code>/lost+found</code> : Recovered Files</strong> – If the file system crashes (It happens for many reasons, power failure, applications are not properly closed, etc,.) the corrupted files will be placed under this directory. File system check will be performed on next boot.</li><li><strong><code>/media</code> : Removable Media</strong> – Temporary mount directory for external removable media/devices (floppies, CDs, DVDs).</li><li><strong><code>/mnt</code> : Temporary Mount Points</strong> – Temporary mount directory, where we can mount filesystems temporarily.</li><li><strong><code>/opt</code> : Optional Packages</strong> – opt stands for optional, Third party applications can be installed under /opt directory, which is not available in official repository or proprietary software.</li><li><strong><code>/proc</code> : Kernel &amp; Process Files</strong> – A virtual filesystem that contains information about running process (/proc/(pid), kernel &amp; system resources (/proc/uptime &amp; /proc/vmstat).</li><li><strong><code>/root</code> : Root Home Directory</strong> – is the superuser’s home directory, which is not same as /.</li><li><strong><code>/run</code> : Application State Files</strong> – is a tmpfs (temporary file system) available early in the boot process, later files get truncated at the beginning of the boot process.</li><li><strong><code>/sbin</code> : System Administration Binaries</strong> – <code>/sbin</code> also contains binary executable similar to /bin but it’s require superuser privilege to perform the commands, which is used for system maintenance purpose.</li><li><strong><code>/selinux</code> : SELinux Virtual File System</strong> – Security-Enhanced Linux (SELinux) is a Linux kernel security module that provides a mechanism for supporting access control security policies, applicable for RPM based systems, such as (RHEL, CentOS, Fedora, Oracle Linux, Scentific Linux &amp; openSUSE).</li><li><strong><code>/srv</code> : Service Data</strong> – srv stands for service, contain data directories of varies services provided by the system such as HTTP (/srv/www/) or FTP(/srv/ftp/)</li><li><strong><code>/sys</code> : virtual filesystem or pseudo file system (sysfs)</strong> – Modern Linux distributions included a /sys directory, since 2.6.X kernels. It provides a set of virtual files by exporting information about various kernel subsystems, hardware devices and associated device drivers from the kernel’s device model to user space.</li><li><strong><code>/tmp</code> : Temporary Directory </strong> – <code>/tmp</code> stands for Temporary (Temporary Files) – Applications store temporary files in the /tmp directory, when its running/required. Which will automatically deleted on next reboot.</li><li><strong><code>/usr</code> : User Binaries</strong> – Contains binaries, libraries, documentation and source-code for second level programs (read-only user data). Command binaries (/usr/bin), system binaries (/usr/sbin), libraries (/usr/lib) for the binaries. source code (/usr/src), documents (/usr/share/doc).</li><li><strong><code>/var</code> : Variable</strong> – var stands for Variable, It contains Application cache files (/var/cache), package manager &amp; database files (/var/lib), lock file (/var/lock), various logs (/var/log), users mailboxes (/var/mail) &amp; print queues and outgoing mail queue (/var/spool)</li></ul><div style="float: none; margin: 10px 0 10px 0; text-align: center;"> </div>Enjoy…)</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-41260961767209792242017-03-07T23:11:00.001+02:002017-03-07T23:11:35.022+02:00Running Asynchronous background Tasks on Linux with Python 3 Flask and Celery<div dir="ltr" style="text-align: left;" trbidi="on">https://techarena51.com/index.php/running-asynchronous-background-tasks-linux-python-3-flask-celery<br /><br /><br /> <div class="post-1751 post type-post status-publish format-standard has-post-thumbnail hentry category-devops-tutorials category-flask-framework-tutorials-and-examples category-linux-system-administrator-tutorials category-python-tutorials tag-asycronous-tasks-on-linux-tutorials tag-debian-linux-tutorials tag-devops-tutorials tag-flask-celery-application-factory-tutorials tag-flask-celery-tutorials tag-flask-tutorials tag-linux-system-administration-tutorials tag-python-web-tutorials tag-rabbitmq-celery-tutorial tag-ubuntu-14-04-tutorials"> <a href="https://techarena51.com/index.php/running-asynchronous-background-tasks-linux-python-3-flask-celery/?utm_source=linuxtoday" rel="nofollow"><img align="left" alt="Running Asynchronous background Tasks on Linux with Python 3 Flask and Celery" class="affiliate-image" src="https://techarena51.com/wp-content/uploads/2016/10/celery-flask-rabbitmq-tutorial.jpg" /></a> <br />In this tutorial I will describe how you can run asynchronous tasks on Linux using Celery an asynchronous task queue manager.<br /> While running scripts on Linux some tasks which take time to complete can be done asynchronously. For example a System Update. With Celery you can run such tasks asynchronously in the background and then fetch the results once the task is complete.<br /> You can use celery in your python script and run it from the command line as well but in this tutorial I will be using Flask a Web framework for Python to show you how you can achieve this through a web application.<br /> Before we start it’s good if you have some familiarity with Flask if not you can quickly read my earlier tutorial on building <a href="http://techarena51.com/index.php/how-to-install-python-3-and-flask-on-linux/" target="_blank">Web Applications on Linux with Flask</a> before you proceed.<br /><div class="techa-post-content" data-id="1777" id="techa-1136219514" style="text-align: center;"><ins class="adsbygoogle" data-ad-client="ca-pub-2310453105507515" data-ad-slot="4335662085" data-adsbygoogle-status="done" style="display: inline-block; height: 250px; width: 300px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 250px; margin: 0; padding: 0; position: relative; visibility: visible; width: 300px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 250px; margin: 0; padding: 0; position: relative; visibility: visible; width: 300px;"></ins></ins></ins></div><strong>This tutorial is for Python 3.4, Flask 0.10, Celery 3.1.23 and rabbitmq-server 3.2.4-1 </strong><br /> To make it easier for you I have generated all the code required for the web interface using <a href="https://github.com/Leo-G/Flask-Scaffold" target="_blank">Flask-Scaffold</a> and<br />uploaded it at <a href="https://github.com/Leo-G/Flask-Celery-Linux" target="_blank">https://github.com/Leo-G/Flask-Celery-Linux</a>. You will just need to clone the code and proceed with the installation and configuration as follows:<br /> <strong>Installation</strong><br /> As described above the first step is to clone the code on your Linux server and install the requirements<br /> <pre>git clone https://github.com/Leo-G/Flask-Celery-Linux<br />cd Flask-Celery-Linux<br />virtualenv -p python3.4 venv-3.4<br />source venv-3.4/bin/activate<br />pip install -r requirements.txt<br />sudo apt-get install rabbitmq-server<br /></pre>Most of the requirements including Flask and Celery will be installed using ‘pip’ however we will need to install RabbitMQ via ‘apt-get’ or your distros default package manager.<br /> <strong>What is RabbitMQ?</strong><br />RabbitMQ is a message broker. Celery uses a message broker like RabbitMQ to mediate between clients and workers. To initiate a task, a client adds a message to the queue, which the broker then delivers to a worker. There are other message brokers as well but RabbitMQ is the recommended broker for Celery.<br /> <strong>Configuration</strong><br /> Configurations are stored in the config.py file. There are two configurations that you will need to add,<br />One is your database details where the state and results of your tasks will be stored and two is<br />the RabbitMQ message broker URL for Celery.<br /> <pre>vim config.py<br /></pre><div><div class="syntaxhighlighter python" id="highlighter_784206"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#You can add either a Postgres or MySQL Database </code></div><div class="line number2 index1 alt1"><code class="python comments">#I am using MySQL for this tutorial</code></div><div class="line number3 index2 alt2"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/config.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/config.py</a></code></div><div class="line number4 index3 alt1">&nbsp;</div><div class="line number5 index4 alt2"><code class="python plain">mysql_db_username </code><code class="python keyword">=</code> <code class="python string">'youruser'</code></div><div class="line number6 index5 alt1"><code class="python plain">mysql_db_password </code><code class="python keyword">=</code> <code class="python string">'yourpass'</code></div><div class="line number7 index6 alt2"><code class="python plain">mysql_db_name </code><code class="python keyword">=</code> <code class="python string">'flask_celery_linux'</code></div><div class="line number8 index7 alt1"><code class="python plain">mysql_db_hostname </code><code class="python keyword">=</code> <code class="python string">'localhost'</code></div><div class="line number9 index8 alt2">&nbsp;</div><div class="line number10 index9 alt1">&nbsp;</div><div class="line number11 index10 alt2"><code class="python plain">SQLALCHEMY_DATABASE_URI </code><code class="python keyword">=</code> <code class="python string">"mysql+<a href="pymysql://">pymysql://</a>{DB_USER}:DB_PASS}@{DB_ADDR}/{DB_NAME}"</code><code class="python plain">.</code><code class="python functions">format</code><code class="python plain">(DB_USER</code><code class="python keyword">=</code><code class="python plain">mysql_db_username,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; DB_PASS</code><code class="python keyword">=</code><code class="python plain">mysql_db_password,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; DB_ADDR</code><code class="python keyword">=</code><code class="python plain">mysql_db_hostname,&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; DB_NAME</code><code class="python keyword">=</code><code class="python plain">mysql_db_name)</code></div><div class="line number12 index11 alt1">&nbsp;</div><div class="line number13 index12 alt2"><code class="python comments">#Celery Message Broker Configuration</code></div><div class="line number14 index13 alt1">&nbsp;</div><div class="line number15 index14 alt2"><code class="python plain">CELERY_BROKER_URL </code><code class="python keyword">=</code> <code class="python string">'<a href="amqp://guest@localhost//">amqp://guest@localhost//</a>'</code></div><div class="line number16 index15 alt1"><code class="python plain">CELERY_RESULT_BACKEND </code><code class="python keyword">=</code> <code class="python string">"db+{}"</code><code class="python plain">.</code><code class="python functions">format</code><code class="python plain">(SQLALCHEMY_DATABASE_URI)</code></div></div></td></tr></tbody></table></div></div><strong>Database Migrations</strong><br /> Run the db.py script to create the database tables<br /> <pre>python db.py db init<br />python db.py db migrate<br />python db.py db upgrade<br /></pre>And finally run the in built web server with<br /> <pre>python run.py<br /></pre>You should be able to see the Web Interface at http://localhost:5000<br /> <a href="http://techarena51.com/wp-content/uploads/2016/10/async-tasks-on-linux.png"><img alt="celery flask tutorial" class="aligncenter size-full wp-image-1760" height="743" src="http://techarena51.com/wp-content/uploads/2016/10/async-tasks-on-linux.png" width="1366" /></a><br /> You will need to create a username and password by clicking on sign up, after which you can login.<br /> <strong>Starting the Celery Worker Process</strong><br /> In a new window/terminal activate the virtual environment and start the celery worker process<br /> <pre>cd Flask-Celery-Linux<br />source venv-3.4/bin/activate<br />celery worker -A celery_worker.celery --loglevel=debug<br /></pre>Now go back to the Web interface and click on Commands –&gt; New. Here you can type in any Linux command and see it run asynchronously.<br /> The video below will show you a live demonstration<br /> <strong>Working</strong><br /> To integrate Celery into your Python script or Web application you first need to create an instance of<br />celery with your application name and the message broker URL.<br /> <div><div class="syntaxhighlighter python" id="highlighter_192748"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#full code at <a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/__init__.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/__init__.py</a></code></div><div class="line number2 index1 alt1"><code class="python keyword">from</code> <code class="python plain">celery </code><code class="python keyword">import</code> <code class="python plain">Celery</code></div><div class="line number3 index2 alt2"><code class="python keyword">from</code> <code class="python plain">config </code><code class="python keyword">import</code> <code class="python plain">CELERY_BROKER_URL</code></div><div class="line number4 index3 alt1">&nbsp;</div><div class="line number5 index4 alt2"><code class="python plain">celery </code><code class="python keyword">=</code> <code class="python plain">Celery(__name__, broker</code><code class="python keyword">=</code><code class="python plain">CELERY_BROKER_URL)</code></div></div></td></tr></tbody></table></div></div>Any task that has to run asynchronously then needs to be wrapped by a Celery decorator<br /> <div><div class="syntaxhighlighter python" id="highlighter_546954"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#complete code at <a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py</a></code></div><div class="line number2 index1 alt1">&nbsp;</div><div class="line number3 index2 alt2"><code class="python keyword">from</code> <code class="python plain">app </code><code class="python keyword">import</code> <code class="python plain">celery</code></div><div class="line number4 index3 alt1">&nbsp;</div><div class="line number5 index4 alt2"><code class="python decorator">@celery</code><code class="python plain">.task</code></div><div class="line number6 index5 alt1"><code class="python keyword">def</code> <code class="python plain">run_command(command):</code></div><div class="line number7 index6 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">cmd </code><code class="python keyword">=</code> <code class="python plain">subprocess.Popen(command,stdout</code><code class="python keyword">=</code><code class="python plain">subprocess.PIPE,stderr</code><code class="python keyword">=</code><code class="python plain">subprocess.PIPE)</code></div><div class="line number8 index7 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">stdout,error </code><code class="python keyword">=</code> <code class="python plain">cmd.communicate()</code></div><div class="line number9 index8 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python keyword">return</code> <code class="python plain">{</code><code class="python string">"result"</code><code class="python plain">:stdout, </code><code class="python string">"error"</code><code class="python plain">:error}</code></div></div></td></tr></tbody></table></div></div>You can then call the task in your python scripts using the ‘delay’ or ‘apply_async’ method as follows:<br /> <div><div class="syntaxhighlighter python" id="highlighter_369019"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py</a></code></div><div class="line number2 index1 alt1">&nbsp;</div><div class="line number3 index2 alt2"><code class="python plain">task </code><code class="python keyword">=</code> <code class="python plain">run_command.delay(command) </code></div></div></td></tr></tbody></table></div></div>The difference between the ‘delay’ and the ‘apply_async()’ method is that the latter allows you to specify a time post which the task will be executed.<br /> <div><div class="syntaxhighlighter python" id="highlighter_946105"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python plain">run_command.apply_async(args</code><code class="python keyword">=</code><code class="python plain">[command], countdown</code><code class="python keyword">=</code><code class="python value">30</code><code class="python plain">)</code></div></div></td></tr></tbody></table></div></div>The above command will be executed on Linux after a 30 second delay.<br /> In order to obtain the task status and result you will need the task id.<br /> <div><div class="syntaxhighlighter python" id="highlighter_241777"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py</a></code></div><div class="line number2 index1 alt1"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/models.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/models.py</a></code></div><div class="line number3 index2 alt2">&nbsp;</div><div class="line number4 index3 alt1"><code class="python plain">task </code><code class="python keyword">=</code> <code class="python plain">run_command.delay(cmd.split()) </code></div><div class="line number5 index4 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">task_id </code><code class="python keyword">=</code> <code class="python plain">task.</code><code class="python functions">id</code></div><div class="line number6 index5 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">task_status </code><code class="python keyword">=</code> <code class="python plain">run_command.AsyncResult(task_id)</code></div><div class="line number7 index6 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">task_state </code><code class="python keyword">=</code> <code class="python plain">task_status.state</code></div><div class="line number8 index7 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">result&nbsp; </code><code class="python keyword">=</code>&nbsp;&nbsp; <code class="python functions">str</code><code class="python plain">(task_status.info)</code></div><div class="line number9 index8 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python comments">#Store results in the database using SQlAlchemy</code></div><div class="line number10 index9 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python keyword">from</code> <code class="python plain">models </code><code class="python keyword">import</code> <code class="python plain">Commands</code></div><div class="line number11 index10 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">command </code><code class="python keyword">=</code> <code class="python plain">Commands(request_dict[</code><code class="python string">'name'</code><code class="python plain">], task_id,&nbsp; task_state,&nbsp; result)</code></div><div class="line number12 index11 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">command.add(command)</code></div></div></td></tr></tbody></table></div></div>Tasks can have different states. Pre-defined states include PENDING, FAILURE and SUCCESS. You can<br />define custom states as well.<br /> Incase a task takes a long time to execute or you want to terminate a task pre-maturely you have to use the ‘revoke’ method.<br /> <div><div class="syntaxhighlighter python" id="highlighter_94594"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/commands/views.py</a></code></div><div class="line number2 index1 alt1"><code class="python plain">run_command.AsyncResult(task_id).revoke(terminate</code><code class="python keyword">=</code><code class="python color1">True</code><code class="python plain">)</code></div></div></td></tr></tbody></table></div></div>Just be sure to pass the terminate flag to it else it will be respawned when a celery worker process restarts.<br /> Finally If you are using Flask Application Factories you will need to instantiate Celery when you create your Flask application.<br /> <div><div class="syntaxhighlighter python" id="highlighter_344365"><table border="0" cellpadding="0" cellspacing="0"><tbody><tr><td class="gutter"><div class="line number1 index0 alt2">1</div><div class="line number2 index1 alt1">2</div><div class="line number3 index2 alt2">3</div><div class="line number4 index3 alt1">4</div><div class="line number5 index4 alt2">5</div><div class="line number6 index5 alt1">6</div><div class="line number7 index6 alt2">7</div><div class="line number8 index7 alt1">8</div><div class="line number9 index8 alt2">9</div><div class="line number10 index9 alt1">10</div><div class="line number11 index10 alt2">11</div><div class="line number12 index11 alt1">12</div><div class="line number13 index12 alt2">13</div><div class="line number14 index13 alt1">14</div><div class="line number15 index14 alt2">15</div><div class="line number16 index15 alt1">16</div><div class="line number17 index16 alt2">17</div><div class="line number18 index17 alt1">18</div><div class="line number19 index18 alt2">19</div><div class="line number20 index19 alt1">20</div><div class="line number21 index20 alt2">21</div></td><td class="code"><div class="container"><div class="line number1 index0 alt2"><code class="python comments">#full code at <a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/__init__.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/app/__init__.py</a></code></div><div class="line number2 index1 alt1">&nbsp;</div><div class="line number3 index2 alt2"><code class="python keyword">def</code> <code class="python plain">create_app(config_filename):</code></div><div class="line number4 index3 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">app </code><code class="python keyword">=</code> <code class="python plain">Flask(__name__, static_folder</code><code class="python keyword">=</code><code class="python string">'templates/static'</code><code class="python plain">)</code></div><div class="line number5 index4 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">app.config.from_object(config_filename)</code></div><div class="line number6 index5 alt1">&nbsp;</div><div class="line number7 index6 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python comments"># Init Flask-SQLAlchemy</code></div><div class="line number8 index7 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python keyword">from</code> <code class="python plain">app.basemodels </code><code class="python keyword">import</code> <code class="python plain">db</code></div><div class="line number9 index8 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">db.init_app(app)</code></div><div class="line number10 index9 alt1">&nbsp;</div><div class="line number11 index10 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">celery.conf.update(app.config)</code></div><div class="line number12 index11 alt1">&nbsp;</div><div class="line number13 index12 alt2"><code class="python comments">#<a href="https://github.com/Leo-G/Flask-Celery-Linux/blob/master/run.py">https://github.com/Leo-G/Flask-Celery-Linux/blob/master/run.py</a></code></div><div class="line number14 index13 alt1"><code class="python keyword">from</code> <code class="python plain">app </code><code class="python keyword">import</code> <code class="python plain">create_app</code></div><div class="line number15 index14 alt2">&nbsp;</div><div class="line number16 index15 alt1"><code class="python plain">app </code><code class="python keyword">=</code> <code class="python plain">create_app(</code><code class="python string">'config'</code><code class="python plain">)</code></div><div class="line number17 index16 alt2">&nbsp;</div><div class="line number18 index17 alt1"><code class="python keyword">if</code> <code class="python plain">__name__ </code><code class="python keyword">=</code><code class="python keyword">=</code> <code class="python string">'__main__'</code><code class="python plain">:</code></div><div class="line number19 index18 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">app.run(host</code><code class="python keyword">=</code><code class="python plain">app.config[</code><code class="python string">'HOST'</code><code class="python plain">],</code></div><div class="line number20 index19 alt1"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">port</code><code class="python keyword">=</code><code class="python plain">app.config[</code><code class="python string">'PORT'</code><code class="python plain">],</code></div><div class="line number21 index20 alt2"><code class="python spaces">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</code><code class="python plain">debug</code><code class="python keyword">=</code><code class="python plain">app.config[</code><code class="python string">'DEBUG'</code><code class="python plain">])</code></div></div></td></tr></tbody></table></div></div>To run celery in the background you can use supervisord.<br /> That’s it for now, if you have any suggestions add them in the comments below<br /> Ref:<br /> http://docs.celeryproject.org/en/latest/userguide/calling.html<br />http://blog.miguelgrinberg.com/post/using-celery-with-flask<br /> Images are not mine and are found on the internet </div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-41611626139018090832017-03-07T22:40:00.000+02:002017-03-07T22:40:18.157+02:00Linux Lexicon: Use Watch Command To Run A Command Every X Seconds<div dir="ltr" style="text-align: left;" trbidi="on">https://fossbytes.com/linux-lexicon-watch-command<br /><br /><em><strong><a data-rel="lightbox-0" href="http://fossbytes.com/wp-content/uploads/2016/10/watch-command-in-linux-1.jpg" title=""><img alt="watch-command-in-linux" class="aligncenter wp-image-65203" height="595" src="http://fossbytes.com/wp-content/uploads/2016/10/watch-command-in-linux-1.jpg" width="1071" /></a>Short Bytes:</strong> Have you ever needed to run a command every couple minutes to check on something? Say you need to watch a RAID rebuild or watch a log in real time, but need to search or filter it first. That takes a lot of specialized tools, one for each task really. But using watch command this can be achieved easily.</em><br /> <div class="su-row"><div class="su-column su-column-size-4-5"><div class="su-column-inner su-clearfix"><span class="su-dropcap su-dropcap-style-flat" style="font-size: 1.5em;">T</span>here is a nifty little command that’s incredibly simple to use, and it’s called <strong>watch</strong>.<div class="code-block code-block-9 ai-viewport-1" style="margin: 8px auto; text-align: center;"> <ins class="adsbygoogle" data-ad-client="ca-pub-7669788327297386" data-ad-slot="5042529352" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins></div>What&nbsp;<strong>watch</strong> does is it runs the command in a loop, but clears the terminal before running it each subsequent time, and additionally, displays the interval, command, and date/time as the first line. The default interval is two seconds, but this can be manually set using the&nbsp;<strong>-n</strong> flag with a lower limit of one-tenth of a second.<br /> Here, below, we run the&nbsp;<strong>free</strong> (a memory usage reporting tool)&nbsp;command every five seconds.<br /> <div class="su-note" style="border-color: #e5e5e5;"><div class="su-note-inner su-clearfix" style="background-color: white; border-color: #ffffff; color: black;">devin@fossbytes$watch&nbsp;-n&nbsp;5 free -m</div></div><div><div class="su-note" style="border-color: #e5e5e5;"><div class="su-note-inner su-clearfix" style="background-color: white; border-color: #ffffff; color: black;">Every&nbsp;5.0s:&nbsp;free&nbsp;-m &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;Sat&nbsp;Sep&nbsp;24&nbsp;13:58:24&nbsp;2016</div><div> <div>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;total&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;used&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;free&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;shared&nbsp;&nbsp;buff/cache&nbsp;&nbsp;&nbsp;available</div><div>&nbsp;&nbsp;&nbsp;&nbsp;Mem: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; 257678 &nbsp; &nbsp; &nbsp;39474 &nbsp; 170916&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4101 &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;47287&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;208519</div><div>&nbsp;&nbsp;&nbsp;&nbsp;Swap: &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;7911&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1218&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6693</div></div></div>As you can see, we were able to pass in the&nbsp;<strong>-m</strong>&nbsp;(display in megabytes) flag to&nbsp;<strong>free</strong> without confusing&nbsp;<strong>watch</strong>. This is because all arguments after the first argument, which is a non-option, are passed to the executed command. This gives you some freedom to pass commands without the need of quotes, though, in the cases where&nbsp;<a href="https://fossbytes.com/linux-lexicon-input-output-pipes-redirection-linux/">piping and redirection</a> are used, quotation marks will be required otherwise the output of&nbsp;<strong>watch</strong> will be what’s piped.<br /> There are many options that can be passed to&nbsp;<strong>watch</strong>, like&nbsp;<strong>-t</strong> to remove the header information, or&nbsp;<strong>-d</strong> to highlight the differences between each interval. Below is the full list according to the documentation.<br /> <div class="su-table"><table style="height: 321px; width: 565px;"><tbody><tr class="su-even"><td>-b, –beep</td><td>beep&nbsp;if&nbsp;command&nbsp;has&nbsp;a&nbsp;non-zero&nbsp;exit</td></tr><tr><td>-c, –color</td><td>interpret&nbsp;ANSI&nbsp;color&nbsp;and&nbsp;style&nbsp;sequences</td></tr><tr class="su-even"><td>-d, –difference</td><td>highlight&nbsp;changes&nbsp;between&nbsp;updates</td></tr><tr><td>-e, –errexit</td><td>exit&nbsp;if&nbsp;command&nbsp;has&nbsp;a&nbsp;non-zero&nbsp;exit</td></tr><tr class="su-even"><td>-g, –chgexit</td><td>exit&nbsp;when&nbsp;output&nbsp;from&nbsp;command&nbsp;changes</td></tr><tr><td>-n, –interval</td><td>seconds&nbsp;to&nbsp;wait&nbsp;between&nbsp;updates</td></tr><tr class="su-even"><td>-p, –precise</td><td>attempt&nbsp;run&nbsp;command&nbsp;in&nbsp;precise&nbsp;intervals</td></tr><tr><td>-t, –no-title</td><td>turn&nbsp;off&nbsp;header</td></tr><tr class="su-even"><td>-x, –exec</td><td>pass&nbsp;command&nbsp;to&nbsp;exec&nbsp;instead&nbsp;of&nbsp;“sh&nbsp;-c”</td></tr><tr><td>-h, –help</td><td>display help and exit</td></tr><tr class="su-even"><td>-v, –version</td><td>output&nbsp;version&nbsp;information&nbsp;and&nbsp;exit</td></tr></tbody></table></div>With these options, it’s easy to see how we can combine&nbsp;<strong>watch</strong> and a little bit of scripting with other tools (or sysadmin-fu as some like to call it) to create complex monitoring tools that are custom tailored to our specific needs.<br /> Show us how you&nbsp;<strong>watch</strong> in the comments below.<br /> <strong>Also Read</strong>:&nbsp;<a href="http://fossbytes.com/linux-lexicon-input-output-pipes-redirection-linux/" target="_blank">Linux Lexicon — Input And Output With Pipes And Redirection In Linux</a><br /> </div></div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-89450938901920930092017-03-07T22:30:00.001+02:002017-03-07T22:30:28.475+02:00Scan Ruby-based apps for security issues with Dawnscanner<div dir="ltr" style="text-align: left;" trbidi="on">https://www.helpnetsecurity.com/2016/10/12/scan-ruby-based-apps-dawnscanner<br /><br /><a href="https://github.com/thesp0nge/dawnscanner" target="_blank">Dawnscanner</a> is an open source static analysis scanner designed to review the security of web applications written in Ruby.<br /> <img alt="scan ruby-based apps" class="aligncenter" src="https://www.helpnetsecurity.com/images/posts/dawnscanner1.jpg" title="Dawnscanner" /><br /> <h3>Dawnscanner’s genesis</h3>Its developer, <a href="https://twitter.com/thesp0nge" target="_blank">Paolo Perego</a>, says that he was motivated to create it back in spring 2013, when he needed a tool to review a number of Sinatra-powered security apps, but couldn’t use the <a href="http://brakemanscanner.org/" target="_blank">Brakeman Scanner</a> as it supports only the testing of Ruby on Rails applications.<br /> “Dawnscanner is not tied to a particular MVC (Model View Controller) framework. It is able to review code of <a href="http://www.sinatrarb.com/" target="_blank">Sinatra</a>, <a href="http://padrinorb.com/" target="_blank">Padrino</a> and Ruby on Rails applications, and we plan to add support for <a href="http://hanamirb.org/" target="_blank">Hanami</a> (formerly Lotus for Ruby) in the future,” he told Help Net Security.<br /> The tool is currently able to perform 230 security checks, covering issues from CVE/OSVDB bulletins and the <a href="https://www.owasp.org/index.php/Ruby_on_Rails_Cheatsheet" target="_blank">OWASP Ruby on Rails security cheatsheet</a>. It is also able to spot security issues related to the Ruby interpreter version developers are using for their projects.<br /> Dawnscanner has no GUI, but has command line flags to help people using it in their own application security pipeline. It provides several formatting options for reporting, and can store scan results in a designated folder so developers can keep a history of security findings. Scan results list found vulnerabilities, and and offer mitigation options for them.<br /> <h3>Short- and long-term plans</h3>Paolo’s plans for the tool are many. He wants to add support for the Hanami framework and pure <a href="http://rack.github.io/" target="_blank">Rack</a> applications, make Dawnscanner able to parse custom code to spot OWASP Top 10 security issues, and achieve a tight GitHub integration, so that the tool is able to consume a GitHub URL as an input parameter, download the report, bundle-install dependencies, and start analyzing the code.<br /> If you notice that these plans contrast with the provided Dawnscanner <a href="https://github.com/thesp0nge/dawnscanner/blob/master/Roadmap.md" target="_blank">development roadmap</a>, be aware that the roadmap is also in need of an update.<br /> Paolo is currently working on changing the way Dawnscanner manages its knowledge base, so that the knowledge base can be updated automatically, and a change in it does not lead to a new Dawnscanner gem release.<br /> <img alt="scan ruby-based apps" class="aligncenter" src="https://www.helpnetsecurity.com/images/posts/dawnscanner2.jpg" title="Dawnscanner" /><br /> <h3>Development challenges</h3>“With a full time job, 2 kids and, well, life, it’s really hard to be always on, pushing new code, fixing bugs and so on. There are periods of time in which I had to put energies on different topics,” he notes.<br /> He’s aware that Dawnscanner is no longer a side project “just for fun”, and that people rely on it for their code production. <br /> “Working on a tool designed to be consumed by a community trained to implement agile software development and to release often is really challenging,” he points out. <br /> “They don’t have much time to spend over security issues not strictly related to their business/product. Dawnscanner (and other security tools) must be proactive, always on the move and they must talk in the developers’ language in order to give pointers and instructions that are easy to consume.”<br /> Another problem he encountered while working on the tool is the general lack of awareness of the importance of signing Ruby gems.<br /> “Dawnscanner is digitally signed, and I believe it’s very important to provide people a means to be sure that they’re using a software version that has not been tampered with by a third party. Some of Dawnscanner’s dependencies are, however, not signed, or have an expired signing certificate, and this makes the Dawnscanner installation (with signature verification) fail,” he explains. Users complain to him about third-party expired certificates, but there’s not much he can do about it.<br /> Paolo is proud of his creation, but knows its limitations – he knows that a code review tool can’t be guaranteed to spot all security issues. He advises developers to manually inspect sensitive code, and follow up static analysis with a full application penetration test, to ensure the detection of security issues at runtime.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-54687386436749644922017-03-07T22:17:00.000+02:002017-03-07T22:17:09.630+02:00Useful Vim editor plugins for software developers - part 1<div dir="ltr" style="text-align: left;" trbidi="on">https://www.howtoforge.com/tutorial/vim-editor-plugins-for-software-developers<br /><br />An improved version of Vi, <a href="http://www.vim.org/" target="_blank">Vim</a>&nbsp;is unarguably one of the most popular command line-based text editors in Linux. Besides being a feature-rich text editor, Vim is also used as an IDE (Integrated Development Environment<span>)&nbsp;</span>by software developers around the world.<br /> What makes Vim really powerful is the fact that it's functionality can be extended through plugins. And needless to say, there exist several Vim plugins that are aimed at enhancing users' programming experience.<br /> Especially for software developers who are new to Vim, and are using the editor for development purposes, we'll be discussing some useful Vim plugins - along with examples - in this tutorial.<br /><div style="background-color: white; float: left; margin: 10px 15px 10px 0px; width: 336px;"> <div id="google_ads_div_howtoforge_com_article_rectangle_a_300x250_ad_wrapper"><div id="google_ads_div_howtoforge_com_article_rectangle_a_300x250_ad_container" style="display: inline-block;"><ins style="border: 0px none; display: inline-table; height: 250px; position: relative; width: 300px;"><ins style="border: 0px none; display: block; height: 250px; position: relative; width: 300px;"></ins></ins></div></div></div><span class="highlight">Please note that all the examples, commands, and instructions mentioned in this tutorial have been tested on Ubuntu 16.04, and the Vim version we've used is 7.4.</span><br /> <h2 id="plugin-installation-setup">Plugin installation setup</h2>Given that the tutorial is aimed at new users, it would be reasonable to assume that they don't know how Vim plugins are installed. So, first up, here are the steps required to complete the installation setup:<br /> <ul><li>Create a directory dubbed <em>.vim</em> in your home directory, and then create two sub-directories named <em>autoload</em> and <em>bundle</em>.</li><li>Then, inside the <em>autoload</em> directory, you need to place a file named <a href="http://www.vim.org/scripts/script.php?script_id=2332" target="_blank"><em>pathogen.vim</em></a>, which you can download from <a href="https://raw.githubusercontent.com/tpope/vim-pathogen/master/autoload/pathogen.vim" target="_blank">here</a>.</li><li>Finally, create a file named <em>.vimrc</em> in your home directory and add the following two lines to it:</li></ul><pre>call pathogen#infect() </pre><pre>call pathogen#helptags()</pre><div style="text-align: center;"><a class="fancybox" href="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/big/vimplugins-vimrc.png" id="img-vimplugins-vimrc"><img alt="Vim plugin installation" height="256" src="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/vimplugins-vimrc.png" width="458" /></a></div>That's it. You are now ready to install Vim plugins.<br /> <strong>Note</strong>: Here we've discussed Vim plugin management using Pathogen. There are other plugin managers available as well - to get started, visit <a href="http://vi.stackexchange.com/questions/388/what-is-the-difference-between-the-vim-plugin-managers" target="_blank">this thread</a>.<br /> Now that we are all set, let's discuss a couple of useful Vim plugins.<br /> <h2 id="vim-tagbar-plugin">Vim Tagbar plugin</h2>First up is the Tagbar plugin. This plugin gives you an overview of the structure of a source file by letting you browse the tags it contains. "It does this by creating a sidebar that displays the ctags-generated tags of the current file, ordered by their scope," the <a href="http://majutsushi.github.io/tagbar/" target="_blank">plug-in's official website</a>&nbsp;says. "This means that for example methods in C++ are displayed under the class they are defined in."<br /> Sounds cool, right? Now, lets see how you can install it.<br /> Tagbar's installation is pretty easy - all you have to do is to run the following two commands:<br /> <div class="command">cd ~/.vim/bundle/</div><div class="command">git clone git://github.com/majutsushi/tagbar</div>After the plugin is installed, it's ready for use. You can test it out by opening a .cpp file in Vim, entering <a href="http://www.tldp.org/LDP/intro-linux/html/sect_06_02.html" target="_blank">the command mode</a>, and running the <strong>:TagbarOpen</strong> command. Following is an example screenshot showing the sidebar (towards right) that comes up when the <strong>:TagbarOpen</strong> Vim command was executed:<br /> <div style="text-align: center;"><a class="fancybox" href="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/big/vimplugins-tagbar-example.png" id="img-vimplugins-tagbar-example"><img alt="Vim tagbar plugin" height="294" src="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/vimplugins-tagbar-example.png" width="550" /></a></div>To close the sidebar, use&nbsp;the <strong>:TagbarClose</strong> command. What's worth mentioning here is that you can use the <strong>:TagbarOpen fj</strong> command to open the sidebar as well as shift control to it. This way, you can easily browse the tags it contains - pressing the Enter key on a tag brings up (and shifts control to) the corresponding function in the source code window on the left.<br /> <div style="text-align: center;"><a class="fancybox" href="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/big/vimplugins-tagbar-ex2.png" id="img-vimplugins-tagbar-ex2"><img alt="TagbarClose and TagbarOpen" height="295" src="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/vimplugins-tagbar-ex2.png" width="550" /></a></div>In case you want to repeatedly open and close the sidebar, you can use the <strong>:TagbarToggle</strong> command instead of using <strong>:TagbarOpen</strong> and <strong>:TagbarClose</strong>, respectively.<br /> If typing these commands seems time consuming to you, then you can create a shortcut for the <strong>:TagbarToggle</strong> command. For example, if you put the following line in your <em>.vimrc</em> file:<br /> <pre>nmap <f8> :TagbarToggle<cr></cr></f8></pre>then you can use the F8 key to toggle the Tagbar plugin window.<br /> Moving on, sometimes you'll observe that certain tags are pre-fixed with a +, -, or <code>#</code> symbol. For example, the following screenshot (taken from the plugin's official website) shows some&nbsp;tags prefixed with a + symbol.<br /> <div style="text-align: center;"><a class="fancybox" href="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/big/vimplugins-tagbar-visibility.png" id="img-vimplugins-tagbar-visibility"><img alt="Toggle Tagbar window" height="370" src="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/vimplugins-tagbar-visibility.png" width="550" /></a></div>These symbols basically depict the visibility information for a particular tag. Specifically, <code>+</code> indicates that the member is public, while - indicates a private member. The <code>#</code> symbol, on the other hand, indicates that the member is protected<code>.</code><br /> <code></code>Following are some of the important points related to Tagbar:<br /> <ul><li><span>The plugin website makes it clear that "Tagbar is not a general-purpose tool for managing </span><code>tags</code><span> files. It only creates the tags it needs on-the-fly in-memory without creating any files.&nbsp;<code>tags</code><span> file management is provided by other plugins.</span>"</span></li><li><span>Vim versions &lt; 7.0.167 have a&nbsp;compatibility issue with Tagbar. "<span>If you are affected by this use this alternate Tagbar download instead: </span><a href="https://github.com/majutsushi/tagbar/zipball/70fix">zip</a>," the website says. "<span>It is on par with version 2.2 but probably won't be updated after that due to the amount of changes required."</span></span></li><li><span><span>If you encounter the error </span></span><em>Tagbar: Exuberant ctags not found!</em>&nbsp;while launching the plugin, then you can fix it by downloading and installing ctags from <a href="http://ctags.sourceforge.net/" target="_blank">here</a>.</li><li>For more information on Tagbar, head <a href="https://github.com/majutsushi/tagbar" target="_blank">here</a>.</li></ul><div style="background-color: white; float: left; margin: 10px 15px 10px 0px; width: 336px;"> <div id="google_ads_div_howtoforge_com_article_rectangle_b_300x250_ad_wrapper"><div id="google_ads_div_howtoforge_com_article_rectangle_b_300x250_ad_container" style="display: inline-block;"> <div data-google-query-id="CLztjM2axdICFQ6iUQodpYgDnw" id="div-gpt-ad-1904646143453-3"> </div></div></div></div><h2 id="vim-delimitmate-plugin">Vim delimitMate Plugin</h2>The next plugin we'll be discussing here is&nbsp;<span>delimitMate. The plugin basically provides&nbsp;<span>insert mode auto-completion for quotes, parens, brackets, and more. </span></span><br /> <span><span>It also offers "<span>some other related features that should make your time in insert mode a little bit easier, like syntax awareness (will not insert the closing delimiter in comments and other configurable regions), and expansions (off by default), and some more," the <a href="https://github.com/Raimondi/delimitMate" target="_blank">plugin's official github page</a>&nbsp;says.</span></span></span><br /> Installation of this plugin is similar to the way we installed the previous one:<br /> <div class="command">cd ~/.vim/bundle/</div><div class="command">git clone git://github.com/Raimondi/delimitMate.git</div>Once the plugin is installed successfully (meaning the above commands are successful), you don't have to do anything else - it loads automatically when the Vim editor is launched.<br /> Now, whenever - while in Vim - you type a double quote, single quote, brace, p<span>arentheses, or bracket, they'll be automatically completed.&nbsp;</span><br /> <span>The delimitMate plugin is configurable. For example, you can extend the list of supported symbols, prevent the plugin from loading automatically,&nbsp;turns off the plugin for certain file types, and more. To learn how to configure delimitMate to do all this (and much more), go through the plugin's detailed documentation, which you can access by running the&nbsp;<strong>:help delimitMate</strong> command.</span><br /> <span>The aforementioned command will split your Vim window horizontally into two, with the upper part containing the said documentation.</span><br /> <div style="text-align: center;"><a class="fancybox" href="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/big/vimplugins-delimitmate-help.png" id="img-vimplugins-delimitmate-help"><img alt="Vim deliMate Plugin" height="398" src="https://www.howtoforge.com/images/vim-editor-plugins-for-software-developers/vimplugins-delimitmate-help.png" width="550" /></a></div><h2 id="conclusion">Conclusion</h2>Of the two plugins mentioned in this article, Tagbar - you'll likely&nbsp;agree - requires comparatively&nbsp;more time&nbsp;to get used to. But once it's setup properly (meaning you have things like shortcut launch keys in place), it's a breeze to use. delimitMate, on the other hand, doesn't require you to remember anything.<br /> The tutorial would have given you an idea how useful Vim plugins can be. Apart from the ones discussed here, there are many more plugins available for software developers. We'll discuss a selected bunch&nbsp;in the&nbsp;next part. Meanwhile, drop in a comment if you use a cool development-related Vim plugin and want others to know about it.<br /> In part 2 of this tutorial&nbsp;series I will cover the <a href="https://www.howtoforge.com/tutorial/vim-editor-plugins-for-software-developers-2-syntastic/">Syntax highlighting plugin Syntastic</a>.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-38812082830745149142017-03-07T01:03:00.001+02:002017-03-07T01:03:28.027+02:00How to use Cloud Explorer with Scality S3 server<div dir="ltr" style="text-align: left;" trbidi="on">https://www.linux-toys.com/?p=945<br /><br />I spent a few weeks searching for an open-source S3 server that I can run at home to test <a href="https://github.com/rusher81572/cloudExplorer">Cloud Explorer</a>. I first came across <a href="https://www.minio.io/">Minio</a> which is an open-source S3 server but I could not get it to work with Cloud Explorer because it had issues resolving bucket names via DNS which is a requirement using the AWS SDK. I then read an article about Scality releasing an open-source S3 <a href="https://github.com/scality/s3">server</a> that you can run inside a Docker <a href="https://hub.docker.com/r/scality/s3server/">image</a>. I was able to get Scality up and running quickly with little effort. In this post, I will explain how I got the Scality S3 server setup and how to use it with Cloud Explorer.<br /> <div align="center"><span id="more-945"></span></div>First, I needed to run the Scality Docker image which was a simple one-liner:<br /> <div class="gist" id="gist41654176"> <div class="gist-file"> <div class="gist-data"> <div class="js-gist-file-update-container js-task-list-container file-box"> <div class="file" id="file-cloud-explorer"> <div class="blob-wrapper data type-text" itemprop="text"> <table class="highlight tab-size js-file-line-container" data-tab-size="8"> <tbody><tr> </tr></tbody></table></div></div></div></div></div></div><div class="gist" id="gist41654176"><div class="gist-file"><div class="gist-data"><div class="js-gist-file-update-container js-task-list-container file-box"><div class="file" id="file-cloud-explorer"><div class="blob-wrapper data type-text" itemprop="text"><table class="highlight tab-size js-file-line-container" data-tab-size="8"><tbody><tr><td class="blob-code blob-code-inner js-file-line" id="file-cloud-explorer-LC1">docker run -d --name s3server -p 8000:8000 scality/s3server</td> </tr></tbody></table></div></div></div></div><div class="gist-meta"> <a href="https://gist.github.com/rusher81572/9a15ec2d2c3a3d6129e606a9f4667dc8/raw/af35d66ba1cc7def724e679746ef2c3746715089/cloud%20explorer" style="float: right;">view raw</a> <a href="https://gist.github.com/rusher81572/9a15ec2d2c3a3d6129e606a9f4667dc8#file-cloud-explorer">cloud explorer</a> hosted with by <a href="https://github.com/">GitHub</a> </div></div></div>Next, I needed to modify /etc/hosts on my laptop to resolve buckets properly with Cloud Explorer. By default, the Scality Docker image resolves to localhost which can be <a href="https://github.com/scality/s3">changed</a>. I appended the bucket names that I will use for this test (test and test2) to the localhost entry in /etc/hosts.<br /> <div class="gist" id="gist41654187"> <div class="gist-file"> <div class="gist-data"> <div class="js-gist-file-update-container js-task-list-container file-box"> <div class="file" id="file-hosts"> <div class="blob-wrapper data type-text" itemprop="text"> <table class="highlight tab-size js-file-line-container" data-tab-size="8"> <tbody><tr> </tr></tbody></table></div></div></div></div></div></div><div class="gist" id="gist41654187"><div class="gist-file"><div class="gist-data"><div class="js-gist-file-update-container js-task-list-container file-box"><div class="file" id="file-hosts"><div class="blob-wrapper data type-text" itemprop="text"><table class="highlight tab-size js-file-line-container" data-tab-size="8"><tbody><tr><td class="blob-code blob-code-inner js-file-line" id="file-hosts-LC1">127.0.0.1 localhost test.localhost test2.localhost</td> </tr></tbody></table></div></div></div></div><div class="gist-meta"> <a href="https://gist.github.com/rusher81572/2e4a38ba693cb5160fe86484615e3808/raw/566fe0d6209ec02f4ffeff52bc753c7896a55f3a/hosts" style="float: right;">view raw</a> <a href="https://gist.github.com/rusher81572/2e4a38ba693cb5160fe86484615e3808#file-hosts">hosts</a> hosted with by <a href="https://github.com/">GitHub</a> </div></div></div><br /> Now I can configure the Scality S3 credentials in Cloud Explorer as shown below. I used the default Access and Secret keys by the Docker image.<br /> <img class="aligncenter size-medium" height="1116" src="http://i.imgur.com/NpNvSNJ.png" width="2177" /><br /> <br /> Now Let’s create a bucket:<br /> <img class="aligncenter size-medium" height="1115" src="http://i.imgur.com/ZyKIsZH.png" width="2170" /><br /> <br /> And there it is!<br /> <img class="aligncenter size-medium" height="1115" src="http://i.imgur.com/ZyKIsZH.png" width="2170" /><br /> <br /> Let’s upload a file to make sure it works:<br /> <img class="alignnone size-medium" height="1120" src="http://i.imgur.com/d7LmEZO.png" width="2180" /><br /> <br /> The file is there!<br /> <img class="alignnone size-medium" height="1332" src="http://i.imgur.com/Zc74vXB.png" width="2398" /><br /> <br /> Now let’s run a performance test just for fun:<br /> <img class="alignnone size-medium" height="1115" src="http://i.imgur.com/ZyKIsZH.png" width="2170" /><br /> It was really cool that Scality released this as open-source. Not all of the Amazon S3 features are supported by Scality such as file versioning and others but I hope that this project continues to be worked on and gains community involvement. &nbsp;After using this for a while, I put the S3 server into production for this site. All of the images that you see for this&nbsp;post are hosted by the S3 server inside a Docker image. Please check it out and let me know how you like Cloud Explorer. Please file any bugs on the GitHub issue <a href="https://github.com/rusher81572/cloudExplorer/issues">tracker</a>.</div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-85847611211004427392017-03-07T00:06:00.003+02:002017-03-07T00:06:57.294+02:00USB Killers - Hardware and Software options to destroy your data (or devices)<div dir="ltr" style="text-align: left;" trbidi="on">https://www.linuxforum.com/threads/usb-killers-hardware-and-software-options-to-destroy-your-data-or-devices.2194<br /><br />Every new computer, whether running Linux or not, has some type of Universal Serial Bus (USB) connector. Most electronics now come with a USB connection of some type from TVs to cars. The time has arrived to worry about what device is being placed into these connectors or even being taken out.<br /><br />There are two types of problems to be dealt with in this scenario. The first is to protect your hardware and the second is to protect your data with software.<br /><br />Before we start, let's look at the USB system overall.<br /><br />The USB hardware started in 1996 and began at a speed around 1.5 Mbps (megabytes per second), whereas today the speed is over 10 Gbps (gigabytes per second). The current estimate is that there are around 15 billion USB devices in the world making a USB device a very common item. The main aspects of USB which make it so convenient are the following:<br /><br /><br /><ul><li><i>Single connector type</i>: USB replaces all the different legacy connectors with one well-defined, standardized USB connector for all USB peripheral devices. Thus eliminating the need for different cables and connectors and simplifying the design of the USB devices. The single connector type allows all USB devices to be connected directly to a standard USB port on a computer.</li></ul><ul><li><i>Hot-swappable</i>: USB devices can be added and removed while the computer is running.<br /><br /></li><li><i>Low-cost implementation: The USB devices are managed by the USB Host which is implemented in the PC, phone, etc. The USB devices do not require a controller built-in so the cost is minimized for USB devices.</i><br /><br /></li><li><i>Plug and Play</i>: Operating Systems (OS) identifies, configures, and loads the appropriate device driver when a USB device is connected.<br /><br /></li><li><i>High performance</i>: USB offers a variety of speeds which are increasing with each update of the USB hardware.<br /><br /></li><li><i>Expandability</i>: In theory, up to 127 different devices may be connected to a single bus.<br /><br /></li><li><i>Bus supplied power</i>: The USB controller supplies power to all connected devices so there is no need for external power to be supplied if the device is low-powered. High-powered devices may still require an external power source.<br /><br /></li><li><i>Easy to use for end user</i>: A single standard connector simplifies the usage of the USB device.</li></ul><b>NOTE:</b> For a detailed listing of USB devices and hubs on your Linux system, use the command from a terminal “lsusb -v”.<br /><br />The USB Host is the main component of the USB system. The Host is usually the PC in to which the USB devices are plugged. The USB Host Controller Interface (HCI) is where the Hardware communicates with the Software.<br /><br />The USB system works as a Master/Slave unit, usually termed Bus Mastering. The USB Host is the Master and controls the Slave (periphery devices) setting up a communication protocol between the Host and all devices. Each Host may have one or more Host Controllers which has a port or multiple ports attached to it. The port or ports on a single Host Controller of the USB Hub called the Root Hub. From these ports, devices and hubs may be attached to create USB Bus. All devices on the USB Bus are a Slave to the Host Controller of the Root Hub, which is the Master. Two devices on the same USB Bus can only communicate directly with each other through a USB Bridge.<br /><br />USB connections consist of four connection points. These points allow for power and data connections. For USB 1.0, a pin provides 5V DC power while another pin provides the return of the power to complete the circuit. USB 3.0 on the other hand provides 20 volts, 5 amps, and 100 watts through the power connection. Keep in mind the power being sent through these ports!<br /><br />To give a little more information, the amps used determines how fast the power can travel through the lines. A higher amperage can allow your phone to charge faster. For example, to use a 1 amp charger on your phone may require an hour to fully charge it. If you were to use a 2 amp charger, then your phone may only take half of an hour.<br /><br /><span style="font-size: 22px;"><b>The Hardware option:</b></span><br /><br />Let's look at the hardware first, the USB Kill. The USB Kill device looks like a regular USB Thumb Drive. It contains a capacitor, which is used to store power. The capacitor is charged to -200V DC. Once charged, the capacitor releases the stored voltage into the USB Port. The voltage may then travel into all parts of the device destroying components along the way until the voltage is dissipated. The capacitor is charged again and releases the burst of power into the system again. This process can occur numerous times in a single second. In a PC, the motherboard can be damaged in three seconds or less.<br /><br /><b>NOTE:</b> USB Kill 1.0 can take up to 5 seconds to cause system damage.<br /><br />What this means is that any hardware which has a USB port can be destroyed with the USB Kill device. Hardware can include PCs, laptops, televisions, phones, etc. The discharge of the voltage is similar to a voltage overload or a static burst such as a nearby lightning strike. Some devices have built-in protection against such power spikes, but some may not to the extent of -200V.<br /><br />Everyone should be wary of using devices which may be found or ones others may try to place into your USB Port.<br /><br />The USB Kill Device can be used over and over on many pieces of hardware.<br /><br /><b>NOTE:</b> Please do not use the device maliciously if you should happen to have one.<br /><br />A USB Killer Shield can be used to protect your hardware from being destroyed by a USB Kill device. A USB Killer Shield has two connectors, one is male and the other female. The male connector is plugged into the hardware and any USB device can be plugged into the female connector. By using the shield, you are protected from a USB Kill device.<br /><br /><b>NOTE:</b> One final piece of information is that Apple devices seem to have a built-in protection from such a device so as not to allow the hardware device to be damaged.<br /><br /><span style="font-size: 22px;"><b>The Software option:</b></span><br /><br />For software, there is the USB Kill program. The script is more for your protection of your data. Keep all folders and files encrypted on your hard drive. Use the USBKill script from <a class="externalLink" href="https://github.com/hephaest0s/usbkill" rel="nofollow" target="_blank">https://github.com/hephaest0s/usbkill</a>. Once you have it on your system, run it with the command “sudo python usbkill.py” or “sudo python3 usbkill.py”. Make sure you have a USB drive in the USB Port. You can connect the drive to your wrist with a strap. If someone swipes your laptop the USB thumb drive will be removed. Once the script detects that the USB Port has had activity, then a special script will run as you have configured. The laptop could be powered off so no one can get back on it without the password. Before the laptop powers off, all data could be deleted, etc. The configuration can also specify USB Drives which will not set off the script when attached or removed.<br /><br />The ability of the script is:<br /><br /><ul><li>Compatible with Linux, *BSD and OS X</li><li>Shutdown the computer when there is USB activity</li><li>Customizable. Define which commands should be executed just before shut down</li><li>Ability to whitelist a USB device</li><li>Ability to change the check interval (default: 250ms)</li><li>Ability to melt the program on shut down</li><li>RAM and swap wiping</li><li>Works with sleep mode (OS X)</li><li>No dependency except secure-delete if you want usbkill to delete files/folders for you or if you want to wipe RAM or swap. sudo apt-get install secure-delete</li><li>Sensible defaults</li></ul>The USB Kill script can help safeguard your data from theft or your hardware from unwanted use to prevent someone from copying the data off your system.<br /><br />Be aware that the USB ports are useful and convenient, but they can pose a risk to your hardware and data. Keep your hardware and data safe as much as possible. </div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-62601597465854610092017-03-06T23:53:00.000+02:002017-03-06T23:53:12.001+02:00Free tool protects PCs from master boot record attacks<div dir="ltr" style="text-align: left;" trbidi="on">http://www.csoonline.com/article/3133115/security/free-tool-protects-pcs-from-master-boot-record-attacks.html<br /><br /><section class="deck viewability"><h2 itemprop="description">The tool acts as a system driver and blocks ransomware and other malicious programs from injecting rogue code into the master boot record</h2><h2 itemprop="description"><img alt="MBRFilter protects Windows computers against MBR attacks." itemprop="contentUrl" src="http://images.techhive.com/images/article/2016/06/laser_protection-100664263-primary.idge.jpg" /></h2><figure class="hero-img" itemprop="image" itemscope="" itemtype="http://schema.org/ImageObject"><figcaption>Credit:<span itemprop="copyrightHolder">Thinkstock </span></figcaption></figure><section class="epo cat-narrow" id="drr-top-ad"> <div class="related-promo-wrapper"><div class="promo apart"><div class="hed">More like this</div><ul class="cso"><li class="clearfix with-image"><a href="http://www.csoonline.com/article/3091710/security/new-satana-ransomware-encrypts-user-files-and-master-boot-record.html"><img alt="security keys locks" class="carousel.idgeImage imgId100603123 " itemprop="image" src="http://core1.staticworld.net/images/idge/imported/imageapi/2015/08/06/19/securitykeys1-100603123-carousel.idge.jpg" /></a><div class="item"><div class="title"><a href="http://www.csoonline.com/article/3091710/security/new-satana-ransomware-encrypts-user-files-and-master-boot-record.html">New Satana ransomware encrypts user files and master boot record</a></div></div></li><li class="clearfix with-image"><a href="http://www.csoonline.com/article/3048319/security/petya-ransomware-overwrites-mbrs-locking-users-out-of-their-computers.html"><img alt="petya ransomware logo" class="carousel.idgeImage imgId100652676 " itemprop="image" src="http://core4.staticworld.net/images/article/2016/03/petya_ransomware_logo_1-100652676-carousel.idge.jpg" /></a><div class="item"><div class="title"><a href="http://www.csoonline.com/article/3048319/security/petya-ransomware-overwrites-mbrs-locking-users-out-of-their-computers.html">Petya ransomware overwrites MBRs, locking users out of their computers</a></div></div></li><li class="clearfix with-image"><a href="http://www.csoonline.com/article/3054592/security/experts-crack-petya-ransomware-enable-hard-drive-decryption-for-free.html"><img alt="petya ransomware logo" class="carousel.idgeImage imgId100652676 " itemprop="image" src="http://core4.staticworld.net/images/article/2016/03/petya_ransomware_logo_1-100652676-carousel.idge.jpg" /></a><div class="item"><div class="title"><a href="http://www.csoonline.com/article/3054592/security/experts-crack-petya-ransomware-enable-hard-drive-decryption-for-free.html">Experts crack Petya ransomware, enable hard drive decryption for free</a></div></div></li><li class="clearfix with-image"> <a href="http://www.csoonline.com/video/74879/ransomware-roundtable-is-this-the-new-spam"><img class="posterImage imgId100711607 " itemprop="image" src="http://core3.staticworld.net/images/article/2017/03/rsa17_csin_014_ransomwareroundtable_v1.00_04_54_04.still001-100711607-poster.jpg" /></a><div class="item"><div class="item-eyebrow">Video</div><div class="title"><a href="http://www.csoonline.com/video/74879/ransomware-roundtable-is-this-the-new-spam">Ransomware roundtable: Is this the new spam?</a></div></div></li></ul></div></div><div class="ad"><div class="adunit ad-container" data-google-query-id="CLPtj7LuwtICFVatUQodIFoHgw" id="topimu"></div></div></section> <div class="ad placement"> </div><div class="lazyload_ad"> </div>Cisco's Talos team has developed an open-source tool that can protect the master boot record of Windows computers from modification by ransomware and other malicious attacks. <br /> The tool, <a href="https://github.com/vrtadmin/MBRFilter/releases/tag/1.0">called MBRFilter</a>, functions as a signed system driver and puts the disk's sector 0 into a read-only state. It is available for both 32-bit and 64-bit Windows versions and its source code has been published on GitHub. <br /> The master boot record (MBR) consists of executable code that's stored in the first sector (sector 0) of a hard disk drive and launches the operating system's boot loader. The MBR also contains information about the disk's partitions and their file systems. <br /> Since the MBR code is executed before the OS itself, it can be abused by malware programs to increase their persistence and gain a head start before antivirus programs. Malware programs that infect the MBR to hide from antivirus programs have historically been known as bootkits -- boot-level rootkits. <br /><aside class="nativo-promo smartphone"> </aside>Microsoft attempted to solve the bootkit problem by implementing cryptographic verification of the bootloader in Windows 8 and later. This feature is known as Secure Boot and is based on the Unified Extensible Firmware Interface (UEFI) -- the modern BIOS. <br /><aside class="fakesidebar"><a href="http://www.csoonline.com/article/3044036/security/how-to-respond-to-ransomware-threats.html#tk.cso-fsb"><strong>ALSO ON CSO:</strong> How to respond to ransomware threats</a></aside>The problem is that Secure Boot does not work on all computers and for all Windows versions and does not support MBR-partitioned disks at all. This means that there are still a large number of computers out there that don't benefit from it and remain vulnerable to MBR attacks. <br /> More recently, ransomware authors have also understood the potential for abusing the MBR in their attacks. For example, <a href="http://www.pcworld.com/article/3046626/security/petya-ransomware-overwrites-mbrs-locking-users-out-of-their-computers.html">the Petya ransomware</a>, which appeared in March, replaces the MBR with malicious code that encrypts the OS partition's master file table (MFT) when the computer is rebooted. <br /> The MFT is a special file on NTFS partitions that contains information about every other file: their name, size and mapping to the hard disk sectors. Encrypting the MFT renders the entire system partition unusable and prevents users from being able to use their computers. <br /><aside class="nativo-promo tablet desktop"> </aside>A second ransomware program that targets the MBR and appeared this year is <a href="http://www.pcworld.idg.com.au/article/602805/new-satana-ransomware-encrypts-user-files-master-boot-record/">called Satana</a>. It doesn't not encrypt the MFT, but encrypts the original MBR code itself and replaces it with its own code which displays a ransom note. <br /> A third ransomware program that modifies the MBR to prevent computers from booting is called HDDCrypter and some researchers believe that it predates both Petya and Satana. <br /> "MBRFilter is a simple disk filter based on Microsoft’s diskperf and classpnp example drivers," the Cisco Talos researchers said in a <a href="http://blog.talosintel.com/2016/10/mbrfilter.html">blog post</a>. "It can be used to prevent malware from writing to Sector 0 on all disk devices connected to a system. Once installed, the system will need to be booted into Safe Mode in order for Sector 0 of the disk to become accessible for modification." <br /><h2 itemprop="description">&nbsp;</h2></section></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-587596444776040732017-03-06T23:40:00.001+02:002017-03-06T23:40:33.351+02:00Browse anonymously in Kali Linux with Anonsurf<div dir="ltr" style="text-align: left;" trbidi="on">https://www.blackmoreops.com/2016/10/17/browse-anonymously-in-kali-linux-with-anonsurf<br /><br />IP spoofing, also known as IP address forgery or a host file hijack, is a hijacking technique in which a cracker masquerades as a trusted host to conceal his identity, spoof a Web site, hijack browsers, or gain access to a network. We use various methods to spoof our IP addresses, most common being using Proxy, VPN and TOR. <a href="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-5.jpg"><img alt="Browse anonymously with Anonsurf in Kali Linux - blackMORE Ops -5" class="alignleft size-full wp-image-6995" height="375" src="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-5.jpg" width="600" /></a>I found this interesting tool named Anonsurf and it will anonymize the entire system under TOR using IPTables. It will also allow you to start and stop i2p as well. That means you can&nbsp;browse anonymously in Kali Linux with Anonsurf running in the background. Anonsurf will run and keep changing IP address every so often or you can simply restart the process to make it grab a new IP address and thus spoofing your own IP address. Sounds good?<br /> Und3rf10w forked ParrotSec’s git and made a version for Kali Linux which is very easy and straight forward to install. His repo contains the sources of both the anonsurf and pandora packages from ParrotSec combined into one. Und3rf10w also made some small modifications to the DNS servers to use of <a href="https://www.blackmoreops.com/2015/03/01/setup-vpn-on-kali-linux/" target="_blank">Private Internet Access </a>(instead of using FrozenDNS) and added some fixes for users who don’t use the resolvconf application. He also removed some functionality such as the GUI and IceWeasel/Firefox in RAM. There’s a installer script which makes it really easy to install it. You can review the installer script to find out more. This forked version should now work with any Debian or Ubuntu system, but this has only been tested to work on a kali-rolling amd64 system. I am also using the same system but users are advised to test and verify it in their own distro. If it works, then you will be able to hide your IP and gain anonymity as long you’re not signed into any website such as Google, Yahoo etc. I wrote a nice long article comparing different methods i.e. <a href="https://www.blackmoreops.com/2016/01/04/advantages-disadvantages-of-using-proxy-vs-vpn-vs-tor/" target="_blank">TOR vs VPN vs Proxy </a>on top of each other.<br /><div class="insert-post-ads"><div style="float: left; margin-right: 25px;"> </div></div><h3>anonsurf</h3>Anonsurf will anonymize the entire system under TOR using IPTables. It will also allow you to start and stop i2p as well.<br /> <blockquote>NOTE: DO NOT run this as <code>service anonsurf $COMMAND</code>. Run this as <code>anonsurf $COMMAND</code></blockquote><a href="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-10.png"><img alt="Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-10" class="size-full wp-image-7001 alignnone" height="417" src="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-10.png" width="556" /></a><br /> <div><h3>Pandora</h3>Pandora automatically overwrites the RAM when the system is shutting down. Pandora can also be ran manually:<br /> <div><pre>pandora bomb</pre></div>NOTE: This will clear the entire system cache, including active SSH tunnels or sessions so perhaps not a good idea to run it while working. It makes the system freeze for sometime (I tried it in a VM).<br /> </div>So here’s how to configure Anonsurf in Kali Linux:<br /> <h2>Download Anonsurf</h2>Clone anonsurf&nbsp; from GIT<br /> <pre>root@kali:~# git clone https://github.com/Und3rf10w/kali-anonsurf.git<br />Cloning into 'kali-anonsurf'...<br />remote: Counting objects: 275, done.<br />remote: Total 275 (delta 0), reused 0 (delta 0), pack-reused 275<br />Receiving objects: 100% (275/275), 163.44 KiB | 75.00 KiB/s, done.<br />Resolving deltas: 100% (79/79), done.<br />Checking connectivity... done.<br />root@kali:~#</pre><a href="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-1.png"><img alt="Browse anonymously with Anonsurf in Kali Linux - blackMORE Ops -1" class="alignnone size-full wp-image-6991" height="243" src="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-1.png" width="647" /></a><br /> Once it’s downloaded, change directory to kali-anonsurf<br /> <pre>root@kali:~# <br />root@kali:~# cd kali-anonsurf/<br />root@kali:~/kali-anonsurf# <br />root@kali:~/kali-anonsurf# ls<br />installer.sh&nbsp; kali-anonsurf-deb-src&nbsp; LICENSE&nbsp; README.md<br />root@kali:~/kali-anonsurf#</pre><h2>Install anonsurf</h2>With the installer script, it’s very straight forward to install anonsurf in Kali Linux.<br /> <pre>root@kali:~/kali-anonsurf# ./installer.sh<br />--2016-10-13 12:36:53--&nbsp; https://geti2p.net/_static/i2p-debian-repo.key.asc<br />Resolving geti2p.net (geti2p.net)... 2a02:180:a:65:2456:6542:1101:1010, 91.143.92.136<br />Connecting to geti2p.net (geti2p.net)|2a02:180:a:65:2456:6542:1101:1010|:443... connected.<br />HTTP request sent, awaiting response... 200 OK<br />Length: 14455 (14K) [text/plain]<br />Saving to: ‘/tmp/i2p-debian-repo.key.asc’<br /><br />/tmp/i2p-debian-rep 100%[===================&gt;]&nbsp; 14.12K&nbsp; 21.6KB/s&nbsp;&nbsp;&nbsp; in 0.7s<br /><--------output-----truncated-------></--------output-----truncated-------></pre><a href="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-2.jpg"><img alt="Browse anonymously with Anonsurf in Kali Linux - blackMORE Ops -2" class="alignnone size-full wp-image-6992" height="391" src="https://www.blackmoreops.com/wp-content/uploads/2016/10/Browse-anonymously-with-Anonsurf-in-Kali-Linux-blackMORE-Ops-2.jpg" width="736" /></a><br /> In Kali Linux, it will automagically update <code>/etc/tor/torrc</code> file and add the following lines:<br /> <pre>VirtualAddrNetwork 10.192.0.0/10<br />AutomapHostsOnResolve 1<br />TransPort 9040<br />SocksPort 9050<br />DNSPort 53<br />RunAsDaemon 1</pre>It also changes your resolver configuration to the following:<br /> <pre>root@kali:~# cat /etc/resolv.conf<br />nameserver 127.0.0.1<br />nameserver 209.222.18.222<br />nameserver 209.222.18.218</pre>If you don’t like using <a href="https://www.blackmoreops.com/2015/03/01/setup-vpn-on-kali-linux/" target="_blank">Private Internet Access</a> DNS, simply the change DNS in the following lines in <code>/etc/init.d/anonsurf </code>script<br /> <pre>&nbsp;&nbsp; &nbsp;echo -e 'nameserver 127.0.0.1\nnameserver 209.222.18.222\nnameserver 209.222.18.218' &gt; /etc/resolv.conf<br />&nbsp;&nbsp; &nbsp;echo -e " $GREEN*$BLUE Modified resolv.conf to use Tor and Private Internet Access DNS"</pre><h2>Start anonsurf</h2>To start anonsurf, and pass all under TOR, simply start anonsurf. It will also start TOR if that already been started:<br /> <pre>root@kali:~# anonsurf start<br />&nbsp;* killing dangerous applications<br />&nbsp;* cleaning some dangerous cache elements<br />[ i ] Stopping IPv6 services:<br />[ i ] Starting anonymous mode:<br />&nbsp;* Tor is not running!&nbsp; starting it&nbsp; for you<br />&nbsp;* Saved iptables rules<br />&nbsp;* Modified resolv.conf to use Tor and Private Internet Access DNS<br />&nbsp;* All traffic was redirected throught Tor<br />[ i ] You are under AnonSurf tunnel<br />root@kali:~#</pre><h2>Find your new Public IP</h2>You can issue the following command to find out your IP adderss<br /> <pre>root@kali:~# anonsurf myip<br />My ip is:<br />1xx.1xx.2xx.1xx</pre><h2>Restart anonsurf</h2>If you want a new IP address, simply restart anonsurf:<br /> <pre>oot@kali:~# anonsurf restart<br />&nbsp;* killing dangerous applications<br />&nbsp;* cleaning some dangerous cache elements<br />[ i ] Stopping anonymous mode:<br />&nbsp;* Deleted all iptables rules<br />&nbsp;* Iptables rules restored<br />[ i ] Reenabling IPv6 services:<br />&nbsp;* Anonymous mode stopped<br />&nbsp;* killing dangerous applications<br />&nbsp;* cleaning some dangerous cache elements<br />[ i ] Stopping IPv6 services:<br />[ i ] Starting anonymous mode:<br />&nbsp;* Tor is not running!&nbsp; starting it&nbsp; for you<br />&nbsp;* Saved iptables rules<br />&nbsp;* Modified resolv.conf to use Tor and Private Internet Access DNS<br />&nbsp;* All traffic was redirected throught Tor<br />[ i ] You are under AnonSurf tunnel</pre>Then simply check your new IP address using the same myip command:<br /> <pre>root@kali:~# anonsurf myip<br />My ip is:<br />1xx.1xx.6x.6x</pre><h2>Stop anonsurf</h2>To stop anonsurf,<br /> <pre>root@kali:~# anonsurf stop<br />&nbsp;* killing dangerous applications<br />&nbsp;* cleaning some dangerous cache elements<br />[ i ] Stopping anonymous mode:<br />&nbsp;* Deleted all iptables rules<br />&nbsp;* Iptables rules restored<br />[ i ] Reenabling IPv6 services:<br />&nbsp;* Anonymous mode stopped</pre><h2>Testing anonymity</h2>First of all, your IP address definitely changed, so there’s no worry on that side. I checked my <a href="https://www.blackmoreops.com/2015/06/14/how-to-get-public-ip-from-linux-terminal/" target="_blank">public IP from command line,</a> using Google and WhatismyIP.&nbsp; This seems to be working and I was able to browse and compared to just TOR, I think it was slightly faster and more responsive.If you think it’s working slow, simply restart anonsurf and chances are you will end up in a faster connection.<br /> The not so obvious thing people doesn’t check is if they are leaking DNS. I usually do it from <a href="http://dnsleak.com/" target="_blank">http://dnsleak.com/</a> as shown on my post from <a href="https://www.blackmoreops.com/2015/03/01/setup-vpn-on-kali-linux/2/" target="_blank">setting up VPN</a>.&nbsp; However, I did not get any results back, so I used <a href="https://www.perfect-privacy.com/dns-leaktest/" target="_blank">https://www.perfect-privacy.com/dns-leaktest/</a> and https://torguard.net/vpn-dns-leak-test.php and they seems to think I am in Netherlands or Belgium … so all good.<br /> You can also check if you’re leaking IPv6 in here: <a href="http://ipv6leak.com/" target="_blank">http://ipv6leak.com/</a><br /> <h2>Conclusion</h2>Those who doesn’t know what ParrotSec OS is, it is another Security OS similar to Kali Linux developed by Parrot Security. I would give them a go if I were you.<br /> Finally I would like to thank <a href="https://www.parrotsec.org/" target="_blank">ParrotSec</a> and <a href="https://github.com/Und3rf10w/kali-anonsurf" target="_blank">Und3rf10w</a> for taking their time and doing it. I am sure many users around the world would like to use it, specially when your country doesn’t allow access to certain Internet resources.<br /> In case ip spoofing in Kali Linux is a requirement for you, try <a href="https://www.blackmoreops.com/2015/12/28/ip-spoofing-in-kali-linux-with-torsocks/" target="_blank">torsocks</a>. It uses SOCKS proxy which is not commonly used, so chances are you will have fast browsing experiences compared to standard TOR settings.<br /> I think I covered most of anonsurf and browsing anonymously part well. Did I make any mistakes? Do you have a suggestion? Let me know. Comments section is open as always and doesn’t require registration or any validation… so do help others and contribute where applicable.<br /> <h3 class="jp-relatedposts-headline"><em>RELATED ARTICLES</em></h3><div class="jp-relatedposts-post jp-relatedposts-post0 jp-relatedposts-post-thumbs" data-post-format="false" data-post-id="5815"><a class="jp-relatedposts-post-a" data-origin="6113" data-position="0" href="https://www.blackmoreops.com/2015/12/28/ip-spoofing-in-kali-linux-with-torsocks/" rel="nofollow" title="Evade monitoring by IP spoofing in Kali Linux with torsocks torsocks allows you to use most applications in a safe way with TOR. It ensures that DNS requests are handled safely and explicitly rejects any traffic other than TCP from the application you're using. In this post we will cover IP spoofing in Kali Linux with torsocks which will allow users…"><img alt="Evade monitoring by IP spoofing in Kali Linux with torsocks" class="jp-relatedposts-post-img" src="https://i0.wp.com/www.blackmoreops.com/wp-content/uploads/2015/12/IP-spoofing-in-Kali-Linux-with-torsocks-blackMORE-Ops-5.jpg?resize=350%2C200&amp;ssl=1" width="350" /></a><h4 class="jp-relatedposts-post-title"><a class="jp-relatedposts-post-a" data-origin="6113" data-position="0" href="https://www.blackmoreops.com/2015/12/28/ip-spoofing-in-kali-linux-with-torsocks/" rel="nofollow" title="Evade monitoring by IP spoofing in Kali Linux with torsocks torsocks allows you to use most applications in a safe way with TOR. It ensures that DNS requests are handled safely and explicitly rejects any traffic other than TCP from the application you're using. In this post we will cover IP spoofing in Kali Linux with torsocks which will allow users…">Evade monitoring by IP spoofing in Kali Linux with torsocks</a></h4></div><div class="jp-relatedposts-post jp-relatedposts-post1 jp-relatedposts-post-thumbs" data-post-format="false" data-post-id="5845"><a class="jp-relatedposts-post-a" data-origin="6113" data-position="1" href="https://www.blackmoreops.com/2015/11/17/install-angry-ip-scanner-on-kali-linux/" rel="nofollow" title="Install Angry IP Scanner on Kali Linux Angry IP Scanner (or simply ipscan) is an open-source and cross-platform network scanner designed to be fast and simple to use. It is a very fast IP address and port scanner. It can scan IP addresses in any range as well as any their ports. It is cross-platform and lightweight.…"><img alt="Install Angry IP Scanner on Kali Linux" class="jp-relatedposts-post-img" src="https://i0.wp.com/www.blackmoreops.com/wp-content/uploads/2015/11/Install-Angry-IP-Scanner-on-Kali-Linux-blackMORE-Ops-3.png?resize=350%2C200&amp;ssl=1" width="350" /></a><h4 class="jp-relatedposts-post-title"><a class="jp-relatedposts-post-a" data-origin="6113" data-position="1" href="https://www.blackmoreops.com/2015/11/17/install-angry-ip-scanner-on-kali-linux/" rel="nofollow" title="Install Angry IP Scanner on Kali Linux Angry IP Scanner (or simply ipscan) is an open-source and cross-platform network scanner designed to be fast and simple to use. It is a very fast IP address and port scanner. It can scan IP addresses in any range as well as any their ports. It is cross-platform and lightweight.…">Install Angry IP Scanner on Kali Linux</a></h4></div><a class="jp-relatedposts-post-a" data-origin="6113" data-position="2" href="https://www.blackmoreops.com/2015/08/19/create-hidden-service-in-tor-like-silk-road-or-darknet/" rel="nofollow" title="Create hidden service in TOR like Silk Road or DarkNet How many of you have heard of a hidden service in TOR like the Silk Road? It’s really not some mystical location on the dark side of the Internet, or darknet. While it does sound a bit fantastic, a hidden service in TOR is just a server that is connected…"><img alt="Create hidden service in TOR like Silk Road or DarkNet" class="jp-relatedposts-post-img" src="https://i0.wp.com/www.blackmoreops.com/wp-content/uploads/2015/08/Create-TOR-hidden-service-like-Silk-Road-or-DarkNet-blackMORE-Ops.jpg?resize=350%2C200&amp;ssl=1" width="350" /></a><h4 class="jp-relatedposts-post-title"><a class="jp-relatedposts-post-a" data-origin="6113" data-position="2" href="https://www.blackmoreops.com/2015/08/19/create-hidden-service-in-tor-like-silk-road-or-darknet/" rel="nofollow" title="Create hidden service in TOR like Silk Road or DarkNet How many of you have heard of a hidden service in TOR like the Silk Road? It’s really not some mystical location on the dark side of the Internet, or darknet. While it does sound a bit fantastic, a hidden service in TOR is just a server that is connected…">Create hidden service in TOR like Silk Road or DarkNet</a></h4></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-54727165865641027842017-03-06T23:29:00.001+02:002017-03-06T23:29:23.495+02:00How To Download A RPM Package With All Dependencies In CentOS<div dir="ltr" style="text-align: left;" trbidi="on">https://www.ostechnix.com/download-rpm-package-dependencies-centos<br /><br /><div class="post-byline"></div><div class="post-format"> <div class="image-container"> <img alt="download a RPM package with all dependencies" class="attachment-thumb-large size-thumb-large wp-post-image tc-smart-loaded" data-lazy-loaded="true" height="340" src="https://www.ostechnix.com/wp-content/plugins/lazy-load/images/1x1.trans.gif" style="display: block;" width="720" /> </div></div><div style="text-align: justify;">The other day I was trying to create a local repository with packages only we use often in CentOS 7. Of course we can download any package using <em><strong>curl</strong></em> or <em><strong>wget</strong></em> commands. These commands however won’t download the required dependencies. You have to spend some time and manually search and download the dependencies required by the package to install. Well, not anymore. In this brief tutorial, I will walk you through how to download a RPM package with all dependencies in two methods. I tested this guide on CentOS 7, although the same steps may work on other RPM based systems such as RHEL, Fedora and Scientific Linux.</div><div style="float: none; margin: 10px 0 10px 0; text-align: center;"> </div><h2 style="text-align: justify;">Method 1 – Download A RPM Package With All Dependencies Using “Downloadonly” plugin</h2><div style="text-align: justify;">We can easily download any RPM package with all&nbsp;dependencies using <strong>“Downloadonly”</strong> plugin for yum command.</div><div style="text-align: justify;">To install Downloadonly plugin, run the following command as root user.</div><pre>yum install yum-plugin-downloadonly</pre><div style="text-align: justify;">Now, run the following command to download a RPM package.</div><pre>yum install --downloadonly <package-name></package-name></pre><div style="text-align: justify;">By default, this command will download and save the packages in&nbsp;<strong>/var/cache/yum/ in rhel-{arch}-channel/packages</strong> location. However, you can download and save the packages in any location of your choice using <em>“–downloaddir”</em> option.</div><pre>yum install --downloadonly --downloaddir=<directory> <package-name></package-name></directory></pre><div style="text-align: justify;"><strong>Example:</strong></div><pre>yum install --downloadonly --downloaddir=/root/mypackages/ httpd</pre><div style="text-align: justify;"><strong>Sample output:</strong></div><pre>Loaded plugins: fastestmirror<br />Loading mirror speeds from cached hostfile<br /> * base: centos.excellmedia.net<br /> * epel: epel.mirror.angkasa.id<br /> * extras: centos.excellmedia.net<br /> * updates: centos.excellmedia.net<br />Resolving Dependencies<br />--&gt; Running transaction check<br />---&gt; Package httpd.x86_64 0:2.4.6-40.el7.centos.4 will be installed<br />--&gt; Processing Dependency: httpd-tools = 2.4.6-40.el7.centos.4 for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: /etc/mime.types for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Running transaction check<br />---&gt; Package apr.x86_64 0:1.4.8-3.el7 will be installed<br />---&gt; Package apr-util.x86_64 0:1.5.2-6.el7 will be installed<br />---&gt; Package httpd-tools.x86_64 0:2.4.6-40.el7.centos.4 will be installed<br />---&gt; Package mailcap.noarch 0:2.1.41-2.el7 will be installed<br />--&gt; Finished Dependency Resolution<br /><br />Dependencies Resolved<br /><br />=======================================================================================================================================<br /> Package Arch Version Repository Size<br />=======================================================================================================================================<br />Installing:<br /> httpd x86_64 2.4.6-40.el7.centos.4 updates 2.7 M<br />Installing for dependencies:<br /> apr x86_64 1.4.8-3.el7 base 103 k<br /> apr-util x86_64 1.5.2-6.el7 base 92 k<br /> httpd-tools x86_64 2.4.6-40.el7.centos.4 updates 83 k<br /> mailcap noarch 2.1.41-2.el7 base 31 k<br /><br />Transaction Summary<br />=======================================================================================================================================<br />Install 1 Package (+4 Dependent packages)<br /><br />Total download size: 3.0 M<br />Installed size: 10 M<br />Background downloading packages, then exiting:<br />(1/5): apr-1.4.8-3.el7.x86_64.rpm | 103 kB 00:00:01 <br />(2/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:01 <br />(3/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:01 <br />(4/5): httpd-tools-2.4.6-40.el7.centos.4.x86_64.rpm | 83 kB 00:00:01 <br />(5/5): httpd-2.4.6-40.el7.centos.4.x86_64.rpm | 2.7 MB 00:00:09 <br />---------------------------------------------------------------------------------------------------------------------------------------<br />Total 331 kB/s | 3.0 MB 00:00:09 <br /><strong>exiting because "Download Only" specified</strong></pre><div style="text-align: justify;"><a class="" data-rel="lightbox-0" href="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_001.png" title=""><img alt="rootserver1_001" class="aligncenter size-full wp-image-5371 tc-smart-loaded" data-lazy-loaded="true" height="732" src="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_001.png" style="display: block;" width="1366" /></a></div><div style="text-align: justify;">Now go the location that you specified in the above command. You will see there the downloaded package with all dependencies. In my case, I have downloaded the packages in <em><strong>/root/mypackages/</strong></em> directory.</div><div style="text-align: justify;">Let us verify the contents.</div><pre>ls /root/mypackages/</pre><div style="text-align: justify;"><strong>Sample output:</strong></div><pre>apr-1.4.8-3.el7.x86_64.rpm<br />apr-util-1.5.2-6.el7.x86_64.rpm<br />httpd-2.4.6-40.el7.centos.4.x86_64.rpm<br />httpd-tools-2.4.6-40.el7.centos.4.x86_64.rpm<br />mailcap-2.1.41-2.el7.noarch.rpm</pre><div style="text-align: justify;"><a class="" data-rel="lightbox-1" href="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_003-1.png" title=""><img alt="rootserver1_003" class="aligncenter size-full wp-image-5372 tc-smart-loaded" data-lazy-loaded="true" height="511" src="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_003-1.png" style="display: block;" width="816" /></a></div><div style="text-align: justify;">As you see in the above output, the package <strong>httpd</strong> has been downloaded with all dependencies.</div><div style="text-align: justify;">Please note that this plugin is applicable for “yum install/yum update” and not for “yum groupinstall”. By default this plugin will download the latest available packages in the repository. You can however download a particular version by specifying the version.</div><div style="float: none; margin: 10px 0 10px 0; text-align: center;"> <ins class="adsbygoogle" data-ad-client="ca-pub-6701402139964678" data-ad-format="auto" data-ad-slot="8447266942" data-adsbygoogle-status="done" style="display: block; height: 90px;"><ins id="aswift_1_expand" style="background-color: transparent; border: none; display: inline-table; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 809px;"><ins id="aswift_1_anchor" style="background-color: transparent; border: none; display: block; height: 90px; margin: 0; padding: 0; position: relative; visibility: visible; width: 809px;"></ins></ins></ins> </div><div style="text-align: justify;"><strong>Example:</strong></div><pre>yum install --downloadonly --downloaddir=/root/mypackages/ <strong>httpd-2.2.6-40.el7</strong></pre><div style="text-align: justify;">Also, you can download multiple packages at once as shown below.</div><pre>yum install --downloadonly --downloaddir=/root/mypackages/ httpd vsftpd</pre><div id="exam_announcement"><b>Download</b> – <a href="http://ostechnix.tradepub.com/free/w_make272/prgm.cgi?a=1" style="text-decoration: underline;" target="_blank"><strong>Free Guide: “10 Easy Ways to Restore Your Linux System”</strong></a><span class="hu-external"></span></div><h2 style="text-align: justify;">Method 2 – Download A RPM Package With All Dependencies&nbsp;Using “Yumdownloader”&nbsp;utility</h2><div style="text-align: justify;"><strong>Yumdownloader</strong>&nbsp;is a simple, yet useful command-line utility that downloads any RPM package along with all required dependencies in one go.</div><div style="text-align: justify;">Install Yumdownloader using the following command as root user.</div><pre>yum install yum-utils</pre><div style="text-align: justify;">Once installed, run the following command to download a package, for example <strong>httpd</strong>.</div><pre>yumdownloader httpd</pre><div style="text-align: justify;">To download packages with all dependencies, use <em>–resolve</em> option:</div><pre>yumdownloader --resolve httpd</pre><div style="text-align: justify;">By default, Yumdownloader will download the packages in the current working directory.</div><div style="text-align: justify;">To download packages along with all dependencies to a specific location, use <em>–destdir</em> option:</div><pre>yumdownloader --resolve --destdir=/root/mypackages/ httpd</pre><div style="text-align: justify;">Or</div><pre>yumdownloader --resolve --destdir /root/mypackages/ httpd</pre><div style="text-align: justify;"><strong>Sample output:</strong></div><pre>Loaded plugins: fastestmirror<br />Loading mirror speeds from cached hostfile<br /> * base: centos.excellmedia.net<br /> * epel: epel.mirror.angkasa.id<br /> * extras: centos.excellmedia.net<br /> * updates: centos.excellmedia.net<br />--&gt; Running transaction check<br />---&gt; Package httpd.x86_64 0:2.4.6-40.el7.centos.4 will be installed<br />--&gt; Processing Dependency: httpd-tools = 2.4.6-40.el7.centos.4 for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: /etc/mime.types for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.4.6-40.el7.centos.4.x86_64<br />--&gt; Running transaction check<br />---&gt; Package apr.x86_64 0:1.4.8-3.el7 will be installed<br />---&gt; Package apr-util.x86_64 0:1.5.2-6.el7 will be installed<br />---&gt; Package httpd-tools.x86_64 0:2.4.6-40.el7.centos.4 will be installed<br />---&gt; Package mailcap.noarch 0:2.1.41-2.el7 will be installed<br />--&gt; Finished Dependency Resolution<br />(1/5): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:01 <br />(2/5): mailcap-2.1.41-2.el7.noarch.rpm | 31 kB 00:00:02 <br />(3/5): apr-1.4.8-3.el7.x86_64.rpm | 103 kB 00:00:02 <br />(4/5): httpd-tools-2.4.6-40.el7.centos.4.x86_64.rpm | 83 kB 00:00:03 <br />(5/5): httpd-2.4.6-40.el7.centos.4.x86_64.rpm | 2.7 MB 00:00:19</pre><div style="text-align: justify;"><a class="" data-rel="lightbox-2" href="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_004-1.png" title=""><img alt="rootserver1_004" class="aligncenter size-full wp-image-5373 tc-smart-loaded" data-lazy-loaded="true" height="732" src="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_004-1.png" style="display: block;" width="1366" /></a></div><div style="text-align: justify;">Let us verify whether packages have been downloaded in the specified location.</div><pre>ls /root/mypackages/</pre><div style="text-align: justify;"><strong>Sample output:</strong></div><pre>apr-1.4.8-3.el7.x86_64.rpm<br />apr-util-1.5.2-6.el7.x86_64.rpm<br />httpd-2.4.6-40.el7.centos.4.x86_64.rpm<br />httpd-tools-2.4.6-40.el7.centos.4.x86_64.rpm<br />mailcap-2.1.41-2.el7.noarch.rpm</pre><div style="text-align: justify;"><a class="" data-rel="lightbox-3" href="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_005.png" title=""><img alt="rootserver1_005" class="aligncenter size-full wp-image-5374 tc-smart-loaded" data-lazy-loaded="true" height="511" src="http://www.ostechnix.com/wp-content/uploads/2016/10/root@server1_005.png" style="display: block;" width="816" /></a></div><div style="text-align: justify;">Unlike “Downloadonly” plugin, Yumdownload can download the packages related to a particular group.</div><pre>yumdownloader "@Development Tools" --resolve --destdir /root/mypackages/</pre><div style="text-align: justify;">Personally, I prefer Yumdownloader over “Downloadonly” plugin for yum. But, both are extremely easy and handy and does the same job.</div><div style="text-align: justify;">That’s all for today. If you find this guide helpful, please share it on your social networks and let others to benefit.</div><div style="text-align: justify;">Cheers!</div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-32831103360194892292017-03-06T23:15:00.000+02:002017-03-06T23:15:12.957+02:00How to run commands on Linux Container (LXD) instance at provision launch time<div dir="ltr" style="text-align: left;" trbidi="on">https://www.cyberciti.biz/faq/run-commands-on-linux-instance-at-launch-using-cloud-init<br /><br /><span class="drop_cap">I</span> would like to perform common automated configuration tasks and run commands/scripts after the LXD instance starts. How to use cloud-init to run commands on my Linux Container (LXD) instance at launch time?<br /> <span id="more-145698"></span><br /> LXD can use the cloud-init directive to run commands or scripts at the first boot cycle when you launch an instance using the lxc command.<br /><center> <ins class="adsbygoogle" data-ad-client="ca-pub-7825705102693166" data-ad-slot="8594278667" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins> </center><h2>What is a cloud-init?</h2>cloud-init handles early initialization of a cloud instance including LXD and Linux containers. By default cloud-init installed in the Ubuntu/CentOS and all other major cloud images. With cloud-init you can configure:<br /><div class="wp-caption aligncenter" id="attachment_145699" style="width: 609px;"><img alt="Sample cloud-init file for lxc/lxd" class="size-full wp-image-145699" height="486" src="https://s0.cyberciti.org/uploads/faq/2016/10/welcome-yml.jpg" width="599" /><div class="wp-caption-text">Sample cloud-init file for lxc/lxd</div></div><ol><li>Hostname</li><li>Update system</li><li>Install additional packages</li><li>Generate ssh private keys</li><li>Install ssh keys to a users .ssh/authorized_keys so they can log in without a password</li><li>Configure static IP or networking</li><li>Include users/groups</li><li>Creating files</li><li>Install and run chef recipes</li><li>Setup and run puppet</li><li>Add apt or yum repositories</li><li>Run commands on first boot</li><li>Disk setup</li><li>Configure RHN subscription and more.</li></ol>Let us get started with an example.<br /><h2>Step 1: Create lxc container</h2>Type the following command to create a Ubuntu LXC container called foo (but do not run the lxc container yet):<br /> <code>$ lxc init ubuntu: foo</code><br /> One can create a CentOS 7 based Linux container too:<br /> <code>$ lxc init images:centos/7/amd64 bar</code><br /> You can apply certain profile too:<br /> <code>$ lxc init images:ubuntu/xenial/amd64 C2 -p staticlanwan</code><br /><h2>Step 2: Create yml cloud-init config file</h2>In this example, I’m going to setup my lxc hostname, update my system, and Install ssh keys to a users .ssh/authorized_keys so they can log in without a password:<br /> <code>$ vi config.xml</code><br /> First line must be <kbd>#cloud-config</kbd>:<br /> <code>#cloud-config</code><br /> Next, I want to run ‘apt-get upgrade’ on first boot to download and install all security updates for my Linux container, so append:<br /> <code># Apply updates using apt<br /> package_upgrade: true</code><br /> Setup hostname and domain name and update /etc/hosts file:<br /> <code># Set hostname<br /> hostname: foo<br /> fqdn: foo.nixcraft.com<br /> manage_etc_hosts: true</code><br /> Run the following commands on first boot. In this case, update sshd to listen only on private IP and reload sshd, append:<br /><pre>#Run command on first boot only<br />bootcmd:<br /> - [sh, -c, "echo 'ListenAddress 192.168.1.100' &gt;&gt; /etc/ssh/sshd_config"]<br /> - systemctl reload ssh<br /></pre>You can install php7 and nginx packages as follows, append:<br /><pre># Install packages<br />packages:<br /> - nginx<br /> - php-common<br /> - php7.0<br /> - php7.0-cli<br /> - php7.0-common<br /> - php7.0-fpm<br /> - php7.0-gd<br /> - php7.0-mysql<br /> - php7.0-opcache<br /> - php-pear<br /></pre>Finally, install a ssh-key for vivek login and add vivek to sudo file too, append:<br /><pre># User setup<br />users:<br /> - name: vivek<br /> ssh-authorized-keys:<br /> - ***insert-your-key-here****<br /> sudo: ['ALL=(ALL) NOPASSWD:ALL']<br /> groups: sudo<br /> shell: /bin/bash</pre>Save and close the file.<br /><h2>Step 3: Pass cloud-init directives to an instance with user data</h2>You need to set a user.user-data variable as follows for foo Linux container:<br /> <code>$ lxc config set foo user.user-data - &lt; config.yml </code><br /> To view your lxc config for foo container, run:<br /> <code>$ lxc config show foo</code><br /> Sample outputs:<br /><pre><small><br />name: foo<br />profiles:<br />- default<br />config:<br /> user.user-data: "#cloud-config\npackage_upgrade: true\n\n#Set hostname\nhostname:<br /> foo\nfqdn: foo.nixcraft.com\nmanage_etc_hosts: true\n\n#Run command on first boot<br /> only\nbootcmd:\n - [sh, -c, \"echo 'ListenAddress 192.168.1.100' &gt;&gt; /etc/ssh/sshd_config\"]\n<br /> - systemctl reload ssh\n \n# Install packages\npackages:\n - nginx\n - php-common\n<br /> - php7.0\n - php7.0-fpm\n - php7.0-gd\n - php7.0-mysql\n\n# User setup\nusers:\n<br /> - name: vivek\n ssh-authorized-keys:\n - ***insert-your-key-here****\n sudo:<br /> ['ALL=(ALL) NOPASSWD:ALL']\n groups: sudo\n shell: /bin/bash\n\n"<br /> volatile.apply_template: create<br /> volatile.base_image: 315bedd32580c3fb79fd2003746245b9fe6a8863fc9dd990c3a2dc90f4930039<br /> volatile.eth0.hwaddr: 00:16:3e:3d:d9:47<br /> volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'<br />devices:<br /> root:<br /> path: /<br /> type: disk<br />ephemeral: false<br /></small></pre><h2>Step 4: Start your container</h2>Type the following command:<br /> <code>$ lxc start foo</code><br /> Wait for 2-5 minutes. To run all above tasks.<br /><h2>Step 5: Verify it</h2>To login to foo LXC, enter:<br /> <code>$ lxc exec foo bash</code><br /> Verify that sshd bind to private IP:<br /> <code>$ netstat -tulpn</code><br /> Verify that packages are installed and system updated:<br /> <code>$ sudo tail -f /var/log/apt/history.log</code><br /><h2>A note about LXD not working with cloud-init</h2>Please note that cloud-init in LXD triggers after network is up. In other words if network defined as DHCP or static but failed to get an IP address may result into hang ups in cloud-init. It will fail without much warning. Set the following command prior to the first container startup as described in step #4:<br /> <code>$ lxc config set foo user.network_mode link-local<br /> $ lxc start foo</code><br /><h2>Log files for LXD</h2>If you are having problems with cloud-init or cloud-config, take look at the following log files:<br /> <code>$ lxc exec foo bash</code><br /> You can see the actual process logs for cloud-init's processing of the configuration file here:<br /> <code># tail -f /var/log/cloud-init.log </code><br /> Output of your commands can be found here:<br /> <code># tail -f /var/log/cloud-init-output.log</code><br /><h2>Do I need to install the cloud-init package on the host server?</h2>No.<br /><h4>References:</h4><ul><li><a href="http://cloudinit.readthedocs.io/en/latest/index.html" target="_blank">Cloud-init documentation</a></li><li><a href="http://cloudinit.readthedocs.io/en/latest/topics/examples.html" target="_blank">Cloud config examples</a></li><li><a href="https://github.com/lxc/lxd/blob/master/doc/configuration.md" target="_blank">LXD key/value configuration information</a></li><li><a href="https://linuxcontainers.org/lxd/" target="_blank">The LXD home page</a></li><li><a href="https://www.ubuntu.com/cloud/lxd" target="_blank">The LXD container hypervisor</a></li><li>See lxc command man page</li></ul></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-66951998163970718132017-03-06T22:49:00.001+02:002017-03-06T22:49:56.088+02:00Open source OSes for the Internet of Things<div dir="ltr" style="text-align: left;" trbidi="on">http://linuxgizmos.com/open-source-oses-for-the-internet-of-things<br /><br />Previous posts in this IoT series have examined frameworks, development hardware, privacy/security issues, and smart hubs. But it all starts with the OS.<br /><span id="more-54567"></span><br /><br />&nbsp; <br /><table align="center" border="1" cellpadding="10" cellspacing="0" style="width: 95%px;"><tbody><tr><td align="center"><b>An Open Source Perspective on the Internet of Things<br />Part 5: Open Source Operating Systems for IoT</b></td></tr></tbody></table><br /> Over the past decade, the majority of new open source OS projects have shifted from the mobile market to the Internet of Things. In this fifth article in our IoT series, we look at the many new open source operating systems that target IoT. Our previous posts have examined <a href="https://www.linux.com/NEWS/21-OPEN-SOURCE-PROJECTS-IOT" target="new">open source IoT frameworks</a>, as well as <a href="https://www.linux.com/news/linux-and-open-source-hardware-iot" target="new">Linux- and open source development hardware for IoT</a> and <a href="https://www.linux.com/news/smart-linux-home-hubs-mix-iot-ai" target="new">consumer smart home devices</a>. But it all starts with the OS.<br /> In addition to exploring new IoT-focused embedded Linux-based distributions, I’ve included a few older lightweight distributions like OpenWrt that have seen renewed uptake in the segment. While the Linux distros are aimed primarily at gateways and hubs, there has been equivalent growth in non-Linux, open source OSes for IoT that can run on microcontroller units (MCUs), and are typically aimed at IoT edge devices.<br /> Keep in mind that almost all OSes these day are claiming some IoT connection, so the list is somewhat arbitrary. The contenders here fulfill most of the following properties: low memory footprint, high power efficiency, a modular and configurable communication stack, and strong support for specific wireless and sensor technologies. Some projects emphasize IoT security, and many of the non-Linux OSes focus on real-time determinism, which is sometimes a requirement in industrial IoT.<br /> I have generally steered clear of Linux distros that are categorized as “lightweight” but are still largely aimed at desktop use or portable USB stick implementations, rather than headless devices. Still, <a href="https://www.linux.com/news/6-excellent-lightweight-linux-distros-x86-and-arm" target="new">lightweight Linux distros</a> such as LXLE or Linux Lite could be good choices for IoT.<br /> The choices were more difficult with non-Linux open source platforms. After all, most lightweight RTOSes can be used for IoT. I focused on the major platforms, or those that seemed to offer the most promise for IoT. Other potential candidates can be found at this <a href="http://www.osrtos.com/" target="new">Open Source RTOS site</a>.<br /> Not included here is <a href="https://developer.microsoft.com/en-us/windows/iot" target="new">Windows 10 for IoT Core</a>, which is free to makers and supports AllJoyn and IoTivity, but is not fully open source. There are also a number of commercial RTOSes that are major players in IoT, such as Micrium’s µC/OS.<br /> <br /><b>Nine Linux-based open source IoT distros</b><br /> <ul><li><b><a href="https://developers.google.com/brillo/" target="new">Brillo</a></b> — In the year since <a href="http://linuxgizmos.com/android-based-brillo-iot-os-arrives-with-hacker-sbc-support/" target="new">Google released Brillo</a>, the lightweight Android-based distro has seen growing adoption among hacker boards such as the Intel Edison and Dragonboard 410c, and even some computer-on-modules. The future of Brillo is tied to Google’s Weave communications protocol, which it requires. Weave brings discovery, provisioning, and authentication functions to Brillo, which can run on as little as 32MB RAM and 128MB flash. </li></ul><ul><li><b><a href="http://www.huawei.com/minisite/iot/en/liteos.html" target="new">Huawei LiteOS</a></b> — Huawei’s LiteOS, which is not to be confused with the <a href="http://lanterns.eecs.utk.edu/software/liteos/" target="new">open source Unix variant</a>, is said to be based on Linux, but it must be a very lean implementation indeed. <a href="http://linuxgizmos.com/os-rumors-googles-brillo-russias-jolla-huaweis-kirin/" target="new">Announced</a> over a year ago, LiteOS is claimed to be deployable as a kernel as small as 10KB. LiteOS ranges from MCU-based devices to Android-compatible applications processors. The customizable OS is touted for its zero configuration, auto-discovery, auto-networking, fast boot, and real-time operation, and it offers extensive wireless support, including LTE and mesh networking. LiteOS is available with Huawei’s Agile IoT Solution, and it drives its <a href="http://www.huawei.com/en/news/2016/6/End-to-End%20NB-IoT-Solution" target="new">Narrow-band IoT (NB-IoT) Solution</a>. </li></ul><ul><li><b>OpenWrt/LEDE/LininoOS/DD-Wrt</b> — The venerable, networking-focused <a href="https://openwrt.org/" target="new">OpenWrt</a> embedded Linux distro has seen a resurgence due to the IoT craze. The lightweight OpenWrt is frequently found on routers and MIPS-based WiFi boards. Earlier spin-offs such as <a href="http://www.dd-wrt.com/site/index" target="new">DD-Wrt</a> and the Arduino-focused <a href="http://www.linino.org/tag/lininoos/" target="new">LininoOS</a> have recently been followed by an outright fork. The <a href="http://linuxgizmos.com/lede-openwrt-fork-promises-greater-openness/" target="new">Linux Embedded Development Environment</a> (LEDE) project promises more transparent governance and predictable release cycles. </li></ul><ul><li><b><a href="https://ostroproject.org/" target="new">Ostro Linux</a></b> — This Yocto Project based distro broke into the limelight in August when Intel chose it for its <a href="http://linuxgizmos.com/intel-debuts-joule-iot-platform-with-ostro-linux-support/" target="new">Intel Joule</a> module, where it runs on the latest quad-core Atom T5700 SoC. Ostro Linux is compliant with IoTivity, supports numerous wireless technologies, and offers a sensor framework. It has a major focus on IoT security, providing OS-, device-, application, and data-level protections, including cryptography and MAC. The distribution is available in headless and media (XT) versions. </li></ul><ul><li><b><a href="https://www.raspbian.org/" target="new">Raspbian</a></b> — There are some other distributions for the Raspberry Pi that are more specifically aimed at IoT, but the quickly maturing Raspbian is still the best. Because it’s the most popular distro for DIY projects on one of the most widely used IoT platforms, developers can call upon numerous projects and tutorials for help. Now that Raspbian supports Node-RED, the visual design tool for Node-JS, we see less reason to opt for the RPi-specific, IoT-focused <a href="http://thethingbox.io/#home" target="new">Thingbox</a>. </li></ul><ul><li><b><a href="https://developer.ubuntu.com/en/snappy/" target="new">Snappy Ubuntu Core</a></b> — Also called Ubuntu Core with Snaps, this embedded version of Ubuntu Core draws upon a Snap package mechanism that Canonical is spinning off as a <a href="https://www.linux.com/news/ubuntu-snappy-based-package-format-aims-bridge-linux-divide" target="new">universal Linux package format</a>, enabling a single binary package to work on “any Linux desktop, server, cloud or device.” Snaps enable Snappy Ubuntu Core to offer transactional rollbacks, secure updates, cloud support, and an app store platform. Snappy requires only a 600MHz CPU and 128MB RAM, but also needs 4GB of flash. It runs on the Pi and other hacker boards, and has appeared on devices including Erle-Copter drones, Dell Edge Gateways, <a href="http://linuxgizmos.com/private-cloud-server-and-iot-gateway-runs-ubuntu-snappy-on-rpi/" target="new">Nextcloud Box</a>, and <a href="http://linuxgizmos.com/open-source-sdr-sbc-runs-snappy-ubuntu-on-cyclone-v/" target="new">LimeSDR</a>. </li></ul><ul><li><b><a href="https://www.tizen.org/ko?langredirect=1" target="new">Tizen</a></b> — Primarily backed by Samsung, the Linux Foundation hosted embedded Linux stack has barely registered in the mobile market. However, it has been widely used in Samsung TVs and smartwatches, including the new <a href="http://linuxgizmos.com/samsungs-14nm-wearable-soc-debuts-on-gear-3-watch/" target="new">Gear S3</a>, and has been sporadically implemented in its cameras and consumer appliances. Tizen can even <a href="https://www.linux.com/news/tizen-30-joins-growing-list-raspberry-pi-2-distributions" target="new">run on the Raspberry Pi</a>. Samsung has begun to integrate Tizen with its SmartThings smart home system, enabling SmartThings control from Samsung TVs. We can also expect more integration with Samsung’s <a href="http://linuxgizmos.com/samsung-debuts-an-artik-iot-cloud-platform-and-ide/" target="new">Artik modules and Artik Cloud</a>. Artik ships with Fedora, but <a href="https://wiki.tizen.org/wiki/Tizen_On_ARTIK" target="new">Tizen 3.0 has recently been ported</a>, along with Ubuntu Core. </li></ul><ul><li><b><a href="http://www.uclinux.org/" target="new">uClinux</a></b> — The venerable, stripped-down uClinux is the only form of Linux that can run on MCUs, and only then on specific Cortex-M3, M4, and -M7 models. uClinux requires MCUs with built-in memory controllers that can use an external DRAM chip to meet its RAM requirements. Now merged into the mainline Linux kernel, uClinux benefits from the extensive wireless support found in Linux. However, newer MCU-oriented OSes such as Mbed are closing the gap quickly on wireless, and are easier to configure. <a href="http://emcraft.com/products" target="new">EmCraft</a> is one of the biggest boosters for uClinux on MCUs, offering a variety of Cortex-M-based modules with uClinux BSPs. </li></ul><ul><li><b><a href="https://www.yoctoproject.org/" target="new">Yocto Project</a></b> — The Linux Foundation’s Yocto Project is not a Linux distro, but an open source collaborative project to provide developers with templates, tools, and methods to create custom embedded stacks. Because you can customize stacks with minimal overhead, it’s frequently used for IoT. Yocto Project forms the basis for most commercial embedded Linux distros, and is part of projects such as Ostro Linux and Qt for Device Creation. Qt is prepping a <a href="http://linuxgizmos.com/qt-lite-adds-2d-renderer-supports-linux-devices-as-small-as-16mb/" target="new">Qt Lite</a> technology for Qt 5.8 that will optimize Device Creation for smaller IoT targets.</li></ul><br /><b>Nine Non-Linux Open Source IoT OSes</b><br /> <ul><li><b><a href="http://mynewt.apache.org/" target="new">Apache Mynewt</a></b> — The open source, wireless savvy Apache Mynewt for 32-bit MCUs was developed by Runtime and hosted by the Apache Software Foundation. The modular Apache Mynewt is touted for its wireless support, precise configurability of concurrent connections, debugging features, and granular power controls. In May, Runtime and Arduino Srl <a href="http://linuxgizmos.com/open-rtos-focuses-on-32-bit-iot-arduinos-with-ble/" target="new">announced</a> that Apache Mynewt would be available for Arduino Srl’s Primo and STAR Otto SBCs. The OS also supports Arduino LLC boards like the Arduino Zero. (Recently, Arduino Srl and Arduino LLC settled their legal differences, announcing <a href="http://linuxgizmos.com/dueling-arduinos-reunite-with-new-arduino-foundation/" target="new">plans to reunite</a> under an Arduino Holding company and Arduino Foundation.) </li></ul><ul><li><b><a href="https://www.mbed.com/en/" target="new">ARM Mbed</a></b> — ARM’s IoT-oriented OS targets tiny, battery-powered IoT endpoints running on Cortex-M MCUs with as little as 8KB of RAM, and has appeared on the <a href="http://linuxgizmos.com/open-source-microbit-sbc-now-available-for-pre-order/" target="new">BBC Micro:bit</a> SBC. Although originally semi-proprietary, single threaded only, and lacking deterministic features, it’s now open sourced under Apache 2.0, and provides multithreading and RTOS support. Unlike many lightweight RTOSes, Mbed was designed with wireless communications in mind, and it recently added Thread support. The OS supports cloud services that can securely extract data via an Mbed Device Connector. Earlier this year, the project launched a Wearable Reference Design. </li></ul><ul><li><b><a href="http://www.contiki-os.org/" target="new">Contiki</a></b> — With its 10KB RAM and 30KB flash requirements, the open source Contiki can’t get as tiny as Tiny OS or RIOT OS, nor does it offer real-time determinism like RIOT and some others. However, the widely used Contiki provides extensive wireless networking support, with an IPv6 stack contributed by Cisco. The OS supplies a comprehensive list of development tools including a dynamic module loading Cooja Network Simulator for debugging wireless networks. Contiki is touted for efficient memory allocation. </li></ul><ul><li><b><a href="http://www.freertos.org/" target="new">FreeRTOS</a></b> — FreeRTOS is coming close to rivaling Linux among embedded development platforms, and it’s particularly popular for developing IoT end devices. FreeRTOS lacks Linux features such as device drivers, user accounts, and advanced networking and memory management. However, it has a far smaller footprint than Linux, not to mention mainstream RTOSes like VxWorks, and it offers an open source GPL license. FreeRTOS can run on under a half kilobyte of RAM and 5-10KB of ROM, although more typically when used with a TCP/IP stack, it’s more like 24KB of RAM and 60KB flash. </li></ul><ul><li><b><a href="https://github.com/fuchsia-mirror" target="new">Fuchsia</a></b> — Google’s latest open source OS was <a href="http://linuxgizmos.com/is-fuschsia-googles-answer-to-samsungs-tizen/" target="new">partially revealed</a> in August, leaving more questions than answers. The fact that Fuchsia has no relation to Linux, but is based on an LK distro designed to compete with MCU-oriented OSes such as FreeRTOS, led many to speculate that it’s an IoT OS. Yet, Fuchsia also supports mobile and laptop computers, so Google may have much broader ambitions for this early-stage project. </li></ul><ul><li><b><a href="http://nuttx.org/" target="new">NuttX</a></b> — The non-restrictive BSD licensed NuttX is known primarily for being the most common RTOS for open source drones running on APM/ArduPilot and PX4 UAV platform, which are collectively part of the Dronecode platform. NuttX is widely used in other resource-constrained embedded systems, as well. Although it supports x86 and Cortex-A5 and -A8 platforms, this POSIX- and ANSI-based OS is primarily aimed at Cortex-M MCUs. NuttX is fully pre-emptible, with fixed priority, round-robin, and sporadic scheduling. The OS is billed as “a tiny Linux work-alike with a much reduced feature set.” </li></ul><ul><li><b><a href="https://www.riot-os.org/" target="new">RIOT OS</a></b> — The 8-year old RIOT OS is known for its efficient power usage and widespread wireless support. RIOT offers hardware requirements of 1.5KB RAM and 5KB of flash that are almost as low as Tiny OS. Yet it also offers features like multi-threading, dynamic memory management, hardware abstraction, partial POSIX compliance, and C++ support, which are more typical of Linux than lightweight RTOSes. Other features include a low interrupt latency of roughly 40 clock cycles, and priority-based scheduling. You can develop under Linux or OS X and deploy to embedded devices using a native port. </li></ul><ul><li><b><a href="http://webs.cs.berkeley.edu/tos/" target="new">TinyOS</a></b> — This mature, open source BSD-licensed OS is about as tiny as you can get, supporting low power consumption on MCU targets “with a few kB of RAM and a few tens of kB of code space.” Written in a C dialect called nesC, the event-driven TinyOS is used by researchers exploring low-power wireless networking, including multi-hop nets. By the project’s own admission, “computationally-intensive applications can be difficult to write.” The project is working on Cortex-M3 support, but for now it’s still designed for lower-end MCUs and radio chips. </li></ul><ul><li><b><a href="https://www.zephyrproject.org/" target="new">Zephyr</a></b> — The Linux Foundation’s lightweight, security-enabled Zephyr RTOS runs on as little as 2-8KB of RAM. Zephyr works on x86, ARM, and ARC systems, but focuses primarily on MCU-based devices with Bluetooth/BLE and 802.15.4 radios like 6LoWPAN. Zephyr is based on Wind River’s <a href="http://linuxgizmos.com/wind-river-launches-helix-cloud-iot-platform-with-rocket-rtos-and-pulsar-linux/" target="new">Rocket OS</a>, which is based on Viper, a stripped-down version of VxWorks. Initial targets include the Arduino Due and Intel’s <a href="http://linuxgizmos.com/arduino-101-gains-faster-compiler-better-curie-support/" target="new">Arduino 101</a>, among others. Zephyr recently appeared on SeeedStudio’s 96Boards IoT Edition <a href="http://linuxgizmos.com/96boards-goes-cortex-m4-with-iot-edition-and-carbon-sbc/" target="new">BLE Carbon</a> SBC, which is supported by a new <a href="http://linuxgizmos.com/linaro-beams-lite-at-internet-of-things-devices/" target="new">Linaro LITE</a> group. </li></ul></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-35253877106621633032016-07-15T18:06:00.000+02:002016-07-15T18:06:56.178+02:0022 open source tools for creatives<div dir="ltr" style="text-align: left;" trbidi="on">https://opensource.com/life/16/7/22-open-source-tools-creatives<br /><br /><div class="panel-pane pane-entity-field pane-node-field-lead-image"> <div class="field field-name-field-lead-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even" rel="schema:contentUrl"><img alt="22 open source tools for creatives" class="image-full-size" height="292" src="https://opensource.com/sites/default/files/styles/image-full-size/public/images/life/OSDC_women_computing_3.png?itok=wM_L2hHK" title="22 open source tools for creatives" width="520" /></div></div></div></div><div class="panel-pane pane-entity-field pane-file-field-file-image-caption"> <div class="field field-name-field-file-image-caption field-type-text-long field-label-inline clearfix"><div class="field-label">Image credits :&nbsp;</div><div class="field-items"><div class="field-item even"><div class="attribution-info"><a data-rapid_p="54" data-track="attributionNameClick" href="https://www.flickr.com/photos/15132846@N00/5640557375/" target="_blank" title="Go to Ray Smith's photostream">Ray Smith</a><br /><div class="view follow-view clear-float" id="yui_3_16_0_rc_1_1_1408631923806_642"><span class="relationship"> </span></div></div></div></div></div></div><div class="panel-pane pane-block pane-service-links-service-links"> <div class="service-links"><a class="twitter-share-button service-links-twitter-widget" data-counturl="https://opensource.com/life/16/7/22-open-source-tools-creatives" href="http://twitter.com/share?url=https%3A//opensource.com/life/16/7/22-open-source-tools-creatives&amp;count=horizontal&amp;via=opensourceway&amp;text=22%20open%20source%20tools%20for%20creatives&amp;counturl=opensource.com/life/16/7/22-open-source-tools-creatives" rel="nofollow" title="Tweet This"><span class="element-invisible"></span></a><span class="IN-widget" style="display: inline-block; line-height: 1; text-align: center; vertical-align: baseline;"><span style="display: inline-block ! important; font-size: 1px ! important; margin: 0px ! important; padding: 0px ! important; text-indent: 0px ! important; vertical-align: baseline ! important;"><span class="IN-right" id="li_ui_li_gen_1468476961159_1-container"><span class="IN-right" id="li_ui_li_gen_1468476961159_1"><span class="IN-right" id="li_ui_li_gen_1468476961159_1-inner"><span class="IN-right" id="li_ui_li_gen_1468476961159_1-content"><br /></span></span></span></span></span></span> </div></div><div class="panel-pane pane-entity-field pane-node-body"> <div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even">Whether it's visuals, audio, writing, or design, there's an open source tool out there to help get the job done.<br /> "It's absolutely possible to go from concept to finished, polished products, using free and open source software," said&nbsp;Jason.<br /> In this lightning talk, Opensource.com community moderator <a href="https://twitter.com/monsterjavaguns" target="_blank">Jason van Gumster</a> shares&nbsp;22 open source tools for creatives:<br /> <ul><li><a href="https://www.blender.org/" target="_blank">Blender</a>: 3D modeling, animation, video editing</li><li><a href="https://inkscape.org/en/" target="_blank">InkScape</a>: Vector graphics</li><li><a href="http://www.gimp.org/" target="_blank">GIMP</a>: Raster image editing</li><li><a href="https://krita.org/en/" target="_blank">Krita</a>: Illustration</li><li><a href="http://www.audacityteam.org/" target="_blank">Audacity</a>: Audio editing</li><li><a href="http://www.videolan.org/vlc/index.html" target="_blank">VLC</a>: Video player</li><li><a href="https://www.scribus.net/" target="_blank">Scribus</a>: Desktop publishing</li><li><a href="https://calibre-ebook.com/" target="_blank">calibre</a> Digital publishing</li><li><a href="https://github.com/Sigil-Ebook/Sigil" target="_blank">SIGIL</a>: Digital publishing</li><li><a href="http://afterwriting.com/" target="_blank">'afterwriting</a>: Screenwriting</li><li><a href="http://www.trelby.org/" target="_blank">Trelby</a>: Screenwriting</li><li><a href="http://mypaint.org/" target="_blank">MyPaint</a>: Illustration</li><li><a href="https://kdenlive.org/" target="_blank">Kdenlive</a>: Video editing</li><li><a href="http://www.openshot.org/" target="_blank">OpenShot</a>: Video editing</li><li><a href="https://www.shotcut.org/" target="_blank">Shotcut</a>: Video editing</li><li><a href="https://natron.fr/" target="_blank">Natron</a>: Compositing and post-processing</li><li><a href="https://ardour.org/" target="_blank">Ardour</a>: Sound mixing and recording</li><li><a href="http://qtractor.sourceforge.net/qtractor-index.html" target="_blank">Qtractor</a>: Sound mixing and recording</li><li><a href="http://www.rosegardenmusic.com/" target="_blank">Rosegarden</a>: Music scoring</li><li><a href="https://musescore.org/" target="_blank">MuseScore</a>: Music scoring</li><li><a href="http://www.hydrogen-music.org/hcms/" target="_blank">Hydrogen</a>: Drum machine</li><li><a href="http://meshlab.sourceforge.net/" target="_blank">Meshlab</a>: Modeling clean-up for 3D printing</li></ul></div></div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-65004610742083535022016-07-10T01:39:00.001+02:002016-07-10T01:39:41.911+02:00How To Read CPUID Instruction For Each CPU on Linux With x86info and cpuid Commands<div dir="ltr" style="text-align: left;" trbidi="on">http://www.cyberciti.biz/faq/linux-cpuid-command-read-cpuid-instruction-on-linux-for-cpu<br /><br /><span class="drop_cap">I</span>s there a CPU-Z like a freeware/open source software that detects the central processing unit (CPU) of a modern personal computer in Linux operating system? How can I get detailed information about the CPU(s) gathered from the CPUID instruction, including the exact model of CPU(s) on Linux operating system?<br /><span id="more-145331"></span><br />There are three programs on Linux operating system that can provide CPUID information and these tools are useful to find out if specific advanced features such as virtualization, extended page tables, encryption and more:<br /><center> <ins class="adsbygoogle" data-ad-client="ca-pub-7825705102693166" data-ad-slot="2404767461" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins> </center><ol><li><kbd><b>lscpu</b></kbd><b> command</b> – Show information on CPU architecture.</li><li><kbd><b>x86info</b></kbd><b> command</b> – Show x86 CPU diagnostics.</li><li><kbd><b>cpuid</b></kbd><b> command</b> – Dump CPUID information for each CPU. This is the closet tool to CPU-Z app on Linux.</li></ol><h2>x86info</h2>x86info is a program which displays a range of information about the CPUs present in an x86 system.<br /><h3>Install x86info on Debian / Ubuntu Linux</h3><code>$ sudo apt-get install x86info</code><br /><h3>Install x86info on Fedora Linux</h3><code>$ sudo dnf install x86info</code><br /><h3>Install x86info on RHEL/SL/CentOS Linux</h3><code>$ sudo yum install x86info</code><br /><h3>Examples</h3>Simply type the following command:<br /><code># x86info</code><br />Sample outputs:<br /><div class="wp-caption aligncenter" id="attachment_145332" style="width: 608px;"><a href="http://www.cyberciti.biz/faq/linux-cpuid-command-read-cpuid-instruction-on-linux-for-cpu/x86info-display-x86-cpu-diagnostics-on-linux/" rel="attachment wp-att-145332"><img alt="Linux x86info Command To Display-x86 CPU Diagnostics Info On Linux" class="size-full wp-image-145332" height="120" src="https://s0.cyberciti.org/uploads/faq/2015/12/x86info-display-x86-CPU-diagnostics-on-linux.jpg" width="598" /></a><br /><div class="wp-caption-text">Fig.01: Linux x86info Command To Display-x86 CPU Diagnostics Info On Linux</div></div><h4>See TLB, cache sizes and cache associativity</h4><code># x86info -c </code><br />Sample outputs:<br /><div class="wp_syntax"><table><tbody><tr><td class="code"><pre class="ini" style="font-family: monospace;">x86info v1.30. Dave Jones 2001-2011<br />Feedback to <davej redhat.com="">.<br />&nbsp;<br />Found 4 identical CPUs<br />Extended Family: 0 Extended Model: 1 Family: 6 Model: 28 Stepping: 10<br />Type: 0 (Original OEM)<br />CPU Model (x86info's best guess): Atom D510<br />Processor name string (BIOS programmed): Intel(R) Atom(TM) CPU D510 @ 1.66GHz<br />&nbsp;<br />Cache info<br /> L1 Instruction cache: 32KB, 8-way associative. 64 byte line size.<br /> L1 Data cache: 24KB, 6-way associative. 64 byte line size. ECC.<br /> L2 cache: 512KB, 8-way associative. 64 byte line size.<br />TLB info<br />Found unknown cache descriptors: 4f 59 ba c0<br />Total processor threads: 4<br />This system has 1 dual-core processor with hyper-threading (2 threads per core) running at an estimated 1.65GHz</davej></pre></td></tr></tbody></table></div><h4>See CPU feature flags like AES/FPU/SSE and more</h4><code># x86info -f</code><br />Sample outputs:<br /><div class="wp_syntax"><table><tbody><tr><td class="code"><pre class="ini" style="font-family: monospace;">x86info v1.30. Dave Jones 2001-2011<br />Feedback to <davej redhat.com="">.<br />&nbsp;<br />Found 4 identical CPUs<br />Extended Family: 0 Extended Model: 1 Family: 6 Model: 28 Stepping: 10<br />Type: 0 (Original OEM)<br />CPU Model (x86info's best guess): Atom D510<br />Processor name string (BIOS programmed): Intel(R) Atom(TM) CPU D510 @ 1.66GHz<br />&nbsp;<br />Feature flags:<br /> fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflsh ds acpi mmx fxsr sse sse2 ss ht tm pbe sse3 dtes64 monitor ds-cpl tm2 ssse3 cx16 xTPR pdcm movbe<br />Extended feature flags:<br /> SYSCALL xd em64t lahf_lm dts<br />Long NOPs supported: yes<br />&nbsp;<br />Total processor threads: 4<br />This system has 1 dual-core processor with hyper-threading (2 threads per core) running at an estimated 1.65GHz</davej></pre></td></tr></tbody></table></div><h4>See MP table showing CPUs BIOS knows about</h4><code># x86info -mp</code><br />Sample outputs:<br /><div class="wp_syntax"><table><tbody><tr><td class="code"><pre class="ini" style="font-family: monospace;">x86info v1.30. Dave Jones 2001-2011<br />Feedback to <davej redhat.com="">.<br />&nbsp;<br />MP Table:<br /># APIC ID Version State Family Model Step Flags<br /># 0 0x14 BSP, usable 6 12 10 0xbfebfbff<br /># 2 0x14 AP, usable 6 12 10 0xbfebfbff<br />.....<br />..</davej></pre></td></tr></tbody></table></div><h4>Show register values from all possible cpuid calls</h4><code># x86info -r</code><br /><pre>....<br />..<br />eax in: 0x00000000, eax = 0000000a ebx = 756e6547 ecx = 6c65746e edx = 49656e69<br />eax in: 0x00000001, eax = 000106ca ebx = 00040800 ecx = 0040e31d edx = bfebfbff<br />eax in: 0x00000002, eax = 4fba5901 ebx = 0e3080c0 ecx = 00000000 edx = 00000000<br />eax in: 0x00000003, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x00000004, eax = 04004121 ebx = 0140003f ecx = 0000003f edx = 00000001<br />eax in: 0x00000005, eax = 00000040 ebx = 00000040 ecx = 00000003 edx = 00000010<br />eax in: 0x00000006, eax = 00000001 ebx = 00000002 ecx = 00000001 edx = 00000000<br />eax in: 0x00000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x00000008, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x00000009, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x0000000a, eax = 07280203 ebx = 00000000 ecx = 00000000 edx = 00000503<br />eax in: 0x80000000, eax = 80000008 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x80000001, eax = 00000000 ebx = 00000000 ecx = 00000001 edx = 20100800<br />eax in: 0x80000002, eax = 20202020 ebx = 20202020 ecx = 746e4920 edx = 52286c65<br />eax in: 0x80000003, eax = 74412029 ebx = 54286d6f ecx = 4320294d edx = 44205550<br />eax in: 0x80000004, eax = 20303135 ebx = 20402020 ecx = 36362e31 edx = 007a4847<br />eax in: 0x80000005, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x80000006, eax = 00000000 ebx = 00000000 ecx = 02006040 edx = 00000000<br />eax in: 0x80000007, eax = 00000000 ebx = 00000000 ecx = 00000000 edx = 00000000<br />eax in: 0x80000008, eax = 00003024 ebx = 00000000 ecx = 00000000 edx = 00000000<br />....<br />..<br /></pre>To see all information, type:<br /><code># x86info -a</code><br /><h2>cpuid</h2>cpuid dumps detailed information about the CPU(s) gathered from the CPUID instruction, and also determines the exact model of CPU(s) from that information. It dumps all information available from the CPUID instruction. The exact collection of information available varies between manufacturers and processors. The following information is available consistently on all modern CPUs:<br /><ol><li>vendor_id</li><li>version information (1/eax)</li><li>miscellaneous (1/ebx)</li><li>feature information (1/ecx)</li></ol><h3>Install cpuid on Debian / Ubuntu Linux</h3><code>$ sudo apt-get install cpuid</code><br /><h3>Install cpuid on Fedora Linux</h3><code>$ sudo dnf install cpuid</code><br /><h3>Install cpuid on RHEL/SL/CentOS Linux</h3><code>$ sudo yum install cpuid</code><br /><h3>Examples</h3>Simply type the following command (this command provides lots of useful information including list of all features in human readable format):<br /><code># cpuid<br /> # cpuid | less<br /> # cpuid | grep 'something'</code><br />Sample outputs:<br /><div class="wp-caption aligncenter" id="attachment_145333" style="width: 608px;"><a href="http://www.cyberciti.biz/faq/linux-cpuid-command-read-cpuid-instruction-on-linux-for-cpu/cpuid-command-dump-cpuid-information/" rel="attachment wp-att-145333"><img alt="Fig.02: Linux cpuid Command To Dump CPUID information" class="size-full wp-image-145333" height="844" src="https://s0.cyberciti.org/uploads/faq/2015/12/cpuid-command-dump-CPUID-information.jpg" width="598" /></a><br /><div class="wp-caption-text">Fig.02: Linux cpuid Command To Dump CPUID information</div></div><h4>Display information only for the first CPU</h4><code># cpuinfo -1</code><br /><h4>Use the CPUID instruction (default and very reliable)</h4><code># cpuinfo -i</code><br /><h4>Use the CPUID kernel module (not seems to be reliable on all combinations of CPU type and kernel version)</h4><code># cpuinfo -k</code><br /><h4>Search for specific CPU feature</h4><code>## Is virtualization supported (see below for flags)? ##<br /> # cpuid -1 | egrep --color -iw 'vmx|svm|ept|vpid|npt|tpr_shadow|vnmi|flexpriority'<br /> <kbd> <span style="color: red;">VMX</span>: virtual machine extensions = true</kbd><br /> ## Is advanced encryption supported? ##<br /> # cpuid -1 | egrep --color -i 'aes|aes-ni'<br /> <kbd> <span style="color: red;">AES</span> instruction = true</kbd></code><br />Some important flags for sysadmins on Linux based system:<br /><ol><li><kbd><b>vmx</b></kbd> – Intel VT-x, basic virtualization.</li><li><kbd><b>svm</b></kbd> – AMD SVM, basic virtualization.</li><li><kbd><b>ept</b></kbd> – Extended Page Tables, an Intel feature to make emulation of guest page tables faster.</li><li><kbd><b>vpid</b></kbd> – VPID, an Intel feature to make expensive TLB flushes unnecessary when context switching between guests.</li><li><kbd><b>npt</b></kbd> – AMD Nested Page Tables, similar to EPT.</li><li><kbd><b>tpr_shadow</b></kbd> and <kbd><b>flexpriority</b></kbd> – Intel feature that reduces calls into the hypervisor when accessing the Task Priority Register, which helps when running certain types of SMP guests.</li><li><kbd><b>vnmi</b></kbd> – Intel Virtual NMI feature which helps with certain sorts of interrupt events in guests.</li></ol><h4>Display information only for the first CPU</h4><code># cpuinfo -1</code><br />Here is complete information about one of cpu:<br /><div class="wp_syntax"><table><tbody><tr><td class="code"><pre class="ini" style="font-family: monospace;">CPU:<br /> <span style="color: #000099;">vendor_id</span> <span style="color: #000066; font-weight: bold;">=</span> <span style="color: #993333;">"GenuineIntel"</span><br /> version information (1/eax):<br /> processor type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> primary processor (0)</span><br /> <span style="color: #000099;">family</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> Intel Pentium Pro/II/III/Celeron/Core/Core 2/Atom, AMD Athlon/Duron, Cyrix M2, VIA C3 (6)</span><br /> <span style="color: #000099;">model</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0xd (13)</span><br /> stepping id <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x7 (7)</span><br /> extended family <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> extended model <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> (simple synth) <span style="color: #000066; font-weight: bold;">=</span> Intel Core i7-3800/3900 (Sandy Bridge-E C2) / Xeon E5-1600/2600 (Sandy Bridge-E C2/M1), 32nm<br /> miscellaneous (1/ebx):<br /> process local APIC physical ID <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3 (3)</span><br /> cpu count <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x20 (32)</span><br /> CLFLUSH line size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x8 (8)</span><br /> brand index <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> brand id <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00 (0): unknown</span><br /> feature information (1/edx):<br /> x87 FPU on chip <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> virtual-8086 mode enhancement <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> debugging extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> page size extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> time stamp counter <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> RDMSR and WRMSR support <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> physical address extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> machine check exception <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> CMPXCHG8B inst. <span style="color: #000066; font-weight: bold;">=</span> true<br /> APIC on chip <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SYSENTER and SYSEXIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> memory type range registers <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> PTE global bit <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> machine check architecture <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> conditional move/compare instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> page attribute table <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> page size extension <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> processor serial number <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> CLFLUSH instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> debug store <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> thermal monitor and clock ctrl <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> MMX Technology <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> FXSAVE/FXRSTOR <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SSE extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SSE2 extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> self snoop <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> hyper-threading / multi-core supported <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> therm. monitor <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> <span style="color: #000099;">IA64</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> pending break event <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> feature information (1/ecx):<br /> PNI/SSE3: Prescott New Instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> PCLMULDQ instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> 64-bit debug store <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> MONITOR/MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> CPL-qualified debug store <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> VMX: virtual machine extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SMX: safer mode extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> Enhanced Intel SpeedStep Technology <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> thermal monitor 2 <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SSSE3 extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> context ID: adaptive or shared L1 data <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> FMA instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> CMPXCHG16B instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> xTPR disable <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> perfmon and debug <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> process context identifiers <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> direct cache access <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SSE4.1 extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> SSE4.2 extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> extended xAPIC support <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> MOVBE instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> POPCNT instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> time stamp counter deadline <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> AES instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> XSAVE/XSTOR states <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> OS-enabled XSAVE/XSTOR <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> AVX: advanced vector extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> F16C half-precision convert instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> RDRAND instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> hypervisor guest status <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> cache and TLB information (2):<br /> 0x5a: data TLB: 2M/4M pages, 4-way, 32 entries<br /> 0x03: data TLB: 4K pages, 4-way, 64 entries<br /> 0x76: instruction TLB: 2M/4M pages, fully, 8 entries<br /> 0xff: cache data is in CPUID 4<br /> 0xb2: instruction TLB: 4K, 4-way, 64 entries<br /> 0xf0: 64 byte prefetching<br /> 0xca: L2 TLB: 4K, 4-way, 512 entries<br /> processor serial number: 0002-06D7-0000-0000-0000-0000<br /> deterministic cache parameters (4):<br /> --- cache 0 ---<br /> cache type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> data cache (1)</span><br /> cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> self-initializing cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> fully associative cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> extra threads sharing this cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> extra processor cores on this die <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0xf (15)</span><br /> system coherency line size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3f (63)</span><br /> physical line partitions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> ways of associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x7 (7)</span><br /> WBINVD/INVD behavior on lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> inclusive to lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> complex cache indexing <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> number of sets - 1 (s) <span style="color: #000066; font-weight: bold;">=</span> 63<br /> --- cache 1 ---<br /> cache type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> instruction cache (2)</span><br /> cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> self-initializing cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> fully associative cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> extra threads sharing this cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> extra processor cores on this die <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0xf (15)</span><br /> system coherency line size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3f (63)</span><br /> physical line partitions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> ways of associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x7 (7)</span><br /> WBINVD/INVD behavior on lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> inclusive to lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> complex cache indexing <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> number of sets - 1 (s) <span style="color: #000066; font-weight: bold;">=</span> 63<br /> --- cache 2 ---<br /> cache type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> unified cache (3)</span><br /> cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> self-initializing cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> fully associative cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> extra threads sharing this cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> extra processor cores on this die <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0xf (15)</span><br /> system coherency line size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3f (63)</span><br /> physical line partitions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> ways of associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x7 (7)</span><br /> WBINVD/INVD behavior on lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> inclusive to lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> complex cache indexing <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> number of sets - 1 (s) <span style="color: #000066; font-weight: bold;">=</span> 511<br /> --- cache 3 ---<br /> cache type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> unified cache (3)</span><br /> cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3 (3)</span><br /> self-initializing cache level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> fully associative cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> extra threads sharing this cache <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1f (31)</span><br /> extra processor cores on this die <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0xf (15)</span><br /> system coherency line size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3f (63)</span><br /> physical line partitions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> ways of associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x13 (19)</span><br /> WBINVD/INVD behavior on lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> inclusive to lower caches <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> complex cache indexing <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> number of sets - 1 (s) <span style="color: #000066; font-weight: bold;">=</span> 16383<br /> MONITOR/MWAIT (5):<br /> smallest monitor-line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x40 (64)<br /> largest monitor-line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x40 (64)<br /> enum of Monitor-MWAIT exts supported <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> supports intrs as break-event for MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> number of C0 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> number of C1 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> number of C2 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> number of C3 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> number of C4 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> number of C5 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> number of C6 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> number of C7 sub C-states using MWAIT <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> Thermal and Power Management Features (6):<br /> digital thermometer <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> Intel Turbo Boost Technology <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> ARAT always running APIC timer <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> PLN power limit notification <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> ECMD extended clock modulation duty <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> PTM package thermal management <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> digital thermometer thresholds <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> ACNT/MCNT supported performance measure <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> ACNT2 available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> performance-energy bias capability <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> extended feature flags (7):<br /> FSGSBASE instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> IA32_TSC_ADJUST MSR supported <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> BMI instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> HLE hardware lock elision <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> AVX2: advanced vector extensions 2 <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> SMEP supervisor mode exec protection <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> BMI2 instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> enhanced REP MOVSB/STOSB <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> INVPCID instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> RTM: restricted transactional memory <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> QM: quality of service monitoring <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> deprecated FPU CS/DS <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> intel memory protection extensions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> AVX512F: AVX-512 foundation instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> RDSEED instruction <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> ADX instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> SMAP: supervisor mode access prevention <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> Intel processor trace <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> AVX512PF: prefetch instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> AVX512ER: exponent &amp; reciprocal instrs <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> AVX512CD: conflict detection instrs <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> SHA instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> <span style="color: #000099;">PREFETCHWT1</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> Direct Cache Access Parameters (9):<br /> PLATFORM_DCA_CAP MSR bits <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 1</span><br /> Architecture Performance Monitoring Features (0xa/eax):<br /> version ID <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3 (3)</span><br /> number of counters per logical processor <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x4 (4)</span><br /> bit width of counter <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x30 (48)</span><br /> length of EBX bit vector <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x7 (7)</span><br /> Architecture Performance Monitoring Features (0xa/ebx):<br /> core cycle event not available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> instruction retired event not available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> reference cycles event not available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> last-level cache ref event not available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> last-level cache miss event not avail <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> branch inst retired event not available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> branch mispred retired event not avail <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> Architecture Performance Monitoring Features (0xa/edx):<br /> number of fixed counters <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x3 (3)</span><br /> bit width of fixed counters <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x30 (48)</span><br /> x2APIC features / processor topology (0xb):<br /> --- level 0 (thread) ---<br /> bits to shift APIC ID to get next <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> logical processors at this level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2 (2)</span><br /> level number <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> level type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> thread (1)</span><br /> extended APIC ID <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 3</span><br /> --- level 1 (core) ---<br /> bits to shift APIC ID to get next <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x5 (5)</span><br /> logical processors at this level <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x10 (16)</span><br /> level number <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x1 (1)</span><br /> level type <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> core (2)</span><br /> extended APIC ID <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 3</span><br /> XSAVE features (0xd/0):<br /> XCR0 lower 32 bits valid bit field mask <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000007</span><br /> bytes required by fields in XCR0 <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000340 (832)</span><br /> bytes required by XSAVE/XRSTOR area <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000340 (832)</span><br /> XCR0 upper 32 bits valid bit field mask <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000000</span><br /> YMM features (0xd/2):<br /> YMM save state byte size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000100 (256)</span><br /> YMM save state byte offset <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000240 (576)</span><br /> LWP features (0xd/0x3e):<br /> LWP save state byte size <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000000 (0)</span><br /> LWP save state byte offset <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x00000000 (0)</span><br /> extended feature flags (0x80000001/edx):<br /> SYSCALL and SYSRET instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> execution disable <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> 1-GB large page support <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> <span style="color: #000099;">RDTSCP</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> 64-bit extensions technology available <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> Intel feature flags (0x80000001/ecx):<br /> LAHF/SAHF supported in 64-bit mode <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> LZCNT advanced bit manipulation <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> 3DNow! PREFETCH/PREFETCHW instructions <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> <span style="color: #000099;">brand</span> <span style="color: #000066; font-weight: bold;">=</span> <span style="color: #993333;">" Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz"</span><br /> L1 TLB/cache information: 2M/4M pages &amp; L1 TLB (0x80000005/eax):<br /> instruction # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> instruction associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> L1 TLB/cache information: 4K pages &amp; L1 TLB (0x80000005/ebx):<br /> instruction # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> instruction associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> L1 data cache information (0x80000005/ecx):<br /> line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> lines per tag <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> <span style="color: #000099;">associativity</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> size (Kb) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> L1 instruction cache information (0x80000005/edx):<br /> line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> lines per tag <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> <span style="color: #000099;">associativity</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> size (Kb) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> L2 TLB/cache information: 2M/4M pages &amp; L2 TLB (0x80000006/eax):<br /> instruction # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> instruction associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> L2 off (0)</span><br /> data # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> L2 off (0)</span><br /> L2 TLB/cache information: 4K pages &amp; L2 TLB (0x80000006/ebx):<br /> instruction # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> instruction associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> L2 off (0)</span><br /> data # entries <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> data associativity <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> L2 off (0)</span><br /> L2 unified cache information (0x80000006/ecx):<br /> line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x40 (64)<br /> lines per tag <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> <span style="color: #000099;">associativity</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 8-way (6)</span><br /> size (Kb) <span style="color: #000066; font-weight: bold;">=</span> 0x100 (256)<br /> L3 cache information (0x80000006/edx):<br /> line size (bytes) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> lines per tag <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> <span style="color: #000099;">associativity</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> L2 off (0)</span><br /> size (in 512Kb units) <span style="color: #000066; font-weight: bold;">=</span> 0x0 (0)<br /> Advanced Power Management Features (0x80000007/edx):<br /> temperature sensing diode <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> frequency ID (FID) control <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> voltage ID (VID) control <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> thermal trip (TTP) <span style="color: #000066; font-weight: bold;">=</span> false<br /> thermal monitor (TM) <span style="color: #000066; font-weight: bold;">=</span> false<br /> software thermal control (STC) <span style="color: #000066; font-weight: bold;">=</span> false<br /> 100 MHz multiplier control <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> hardware P-State control <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> false</span><br /> <span style="color: #000099;">TscInvariant</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> true</span><br /> Physical Address and Linear Address Size (0x80000008/eax):<br /> maximum physical address bits <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x2e (46)</span><br /> maximum linear (virtual) address bits <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x30 (48)</span><br /> maximum guest physical address bits <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> Logical CPU cores (0x80000008/ecx):<br /> number of CPU cores - 1 <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> <span style="color: #000099;">ApicIdCoreIdSize</span> <span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;"> 0x0 (0)</span><br /> (multi-processing synth): multi-core (c<span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;">8), hyper-threaded (t<span style="color: #000066; font-weight: bold;">=</span>2)</span><br /> (multi-processing method): Intel leaf 0xb<br /> (APIC widths synth): CORE_width<span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;">5 SMT_width=1</span><br /> (APIC synth): PKG_ID<span style="color: #000066; font-weight: bold;">=</span><span style="color: #660066;">0 CORE_ID=1 SMT_ID=1</span><br /> (synth) <span style="color: #000066; font-weight: bold;">=</span> Intel Xeon E5-1600/2600 (Sandy Bridge-E C2/M1), 32nm</pre></td></tr></tbody></table></div><h2>lscpu command example</h2>You will <a href="http://www.cyberciti.biz/faq/lscpu-command-find-out-cpu-architecture-information/">get information about your CPU Architecture</a> on Linux:<br /><code>$ lscpu</code><br />Sample outputs:<br /><pre>Architecture: x86_64<br />CPU op-mode(s): 32-bit, 64-bit<br />Byte Order: Little Endian<br />CPU(s): 32<br />On-line CPU(s) list: 0-31<br />Thread(s) per core: 2<br />Core(s) per socket: 8<br />Socket(s): 2<br />NUMA node(s): 2<br />Vendor ID: GenuineIntel<br />CPU family: 6<br />Model: 45<br />Stepping: 7<br />CPU MHz: 2000.063<br />BogoMIPS: 4001.39<br />Virtualization: VT-x<br />L1d cache: 32K<br />L1i cache: 32K<br />L2 cache: 256K<br />L3 cache: 20480K<br />NUMA node0 CPU(s): 0-7,16-23<br />NUMA node1 CPU(s): 8-15,24-31</pre>Of course you can also extract information from <kbd>/proc/cpuinfo</kbd> and <kbd>/dev/cpu/*</kbd> files:<br /><code>$ less /proc/cpuinfo</code></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-806593380728985892016-07-10T01:26:00.002+02:002016-07-10T01:26:09.383+02:00How to back up and restore file permissions on Linux<div dir="ltr" style="text-align: left;" trbidi="on">http://ask.xmodulo.com/backup-restore-file-permissions-linux.html<br /><br /><div class="question"><b>Question:</b> I want to back up the file permissions of the local filesystem, so that if I accidentally mess up the file permissions, I can restore them to the original state. Is there an easy way to back up and restore file permissions on Linux? </div>You may have heard of a tragic mistake of a rookie sysadmin who accidentally typed "chmod -R 777 /" and wreaked havoc to his/her Linux system. Sure, there are backup tools (e.g., <tt>cp</tt>, <tt>rsync</tt>, <a href="http://xmodulo.com/how-to-version-control-etc-directory-in-linux.html" target="_blank"><tt>etckeeper</tt></a>) which can back up files along with their file permissions. If you are using such backup tools, no worries about corrupted file permissions.<br />But there are cases where you want to temporarily back up file permissions alone (not files themselves). For example, you want to prevent the content of some directory from being overwritten, so you temporarily remove write permission on all the files under the directory. Or you are in the middle of troubleshooting file permission issues, so running <tt>chmod</tt> on files here and there. In these cases, it will be nice to be able to back up the original file permissions before the change, so that you can recover the original file permissions later when needed. In many cases, full file backup is an overkill when all you really want is to back up file permissions.<br />On Linux, it is actually straightforward to back up and restore file permissions using <a href="http://xmodulo.com/configure-access-control-lists-acls-linux.html" target="_blank">access control list (ACL)</a>. The ACL defines access permissions on individual files by different owners and groups on a POSIX-compliant filesystem.<br />Here is <b>how to back up and restore file permissions on Linux using ACL tools</b>.<br />First of all, make sure that you have ACL tools installed.<br />On Debian, Ubuntu or Linux Mint:<br /><div class="console">$ sudo apt-get install acl </div>On CentOS, Fedora or RHEL:<br /><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins><div class="console">$ sudo yum install acl </div><b>To back up the file permissions of all the files in the current directory (and all its sub directories recursively)</b>, run the following command.<br /><div class="console">$ getfacl -R . &gt; permissions.txt </div>This command will export ACL information of all the files into a text file named <tt>permissions.txt</tt>.<br /><a href="https://www.flickr.com/photos/xmodulo/24164605239/" rel="nofollow" target="_blank"><img class="lazy" data-lazy-src="https://c2.staticflickr.com/2/1553/24164605239_e2af4385a0_c.jpg" data-lazy-type="image" src="https://c2.staticflickr.com/2/1553/24164605239_e2af4385a0_c.jpg" /></a><br />For example, the following is a snippet of <tt>permissions.txt</tt> generated from the directory shown in the screenshot.<br /><pre># file: .<br /># owner: dan<br /># group: dan<br />user::rwx<br />group::rwx<br />other::r-x<br /><br /># file: tcpping<br /># owner: dan<br /># group: dan<br /># flags: s--<br />user::rwx<br />group::rwx<br />other::r-x<br /><br /># file: uda20-build17_1.ova<br /># owner: dan<br /># group: dan<br />user::rw-<br />group::rw-<br />other::r--<br /></pre>Now go ahead and change the file permissions as you want. For example:<br /><div class="console">$ chmod -R a-w . </div><b>To restore the original file permissions</b>, go to the directory where <tt>permissions.txt</tt> was generated, and simply run:<br /><div class="console">$ setfacl --restore=permissions.txt </div>Verify that the original file permissions have been restored.<br /><span style="float: left; font-size: 16px; font-weight: bold;">Download this article as ad-free PDF (made possible by <a href="https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&amp;hosted_button_id=PBHS9R4MB9RX4" rel="nofollow" target="_blank">your kind donation</a>):&nbsp;</span><a class="wpptopdfenh" href="http://ask.xmodulo.com/backup-restore-file-permissions-linux.html?format=pdf" rel="noindex,nofollow" target="_blank" title="Download PDF"><img alt="Download PDF" class="lazy" data-lazy-src="http://d2l38su6p74cr6.cloudfront.net/wp-content/plugins/wp-post-to-pdf-enhanced/asset/images/pdf.png" data-lazy-type="image" src="https://d2l38su6p74cr6.cloudfront.net/wp-content/plugins/wp-post-to-pdf-enhanced/asset/images/pdf.png" /></a></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-29753302182171393692016-07-10T01:24:00.003+02:002016-07-10T01:24:31.512+02:00Find Out If Patch Number ( CVE ) Has Been Applied To RHEL / CentOS Linux<div dir="ltr" style="text-align: left;" trbidi="on">http://www.cyberciti.biz/faq/linux-find-out-patch-can-cve-applied<br /><br /><span class="drop_cap">I</span> know how to update my system using the <a href="http://www.cyberciti.biz/faq/rhel-centos-fedora-linux-yum-command-howto/" title="See Linux/Unix yum command examples for more info">yum command</a>. But, how can I find out that patch has been applied to a package? How do I search CVE patch number applied to a package under a Red Hat Enterprise Linux/CentOS/RHEL/Fedora Linux based system?<br /> <span id="more-1951"></span><br /> You need to use the rpm command. Each rpm package stores information about patches including date, small description and CVE number. You can use the <kbd>-q</kbd> query option to display change information for the package.<br /><center> <ins class="adsbygoogle" data-ad-client="ca-pub-7825705102693166" data-ad-slot="2404767461" data-adsbygoogle-status="done" style="display: inline-block; height: 280px; width: 336px;"><ins id="aswift_0_expand" style="background-color: transparent; border: none; display: inline-table; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"><ins id="aswift_0_anchor" style="background-color: transparent; border: none; display: block; height: 280px; margin: 0; padding: 0; position: relative; visibility: visible; width: 336px;"></ins></ins></ins> </center><h2>rpm –changelog option</h2>Use the command as follows:<br /> <code>rpm -q --changelog {<span style="color: #ff9900;">package-name</span>}<br /> rpm -q --changelog {<span style="color: #ff9900;">package-name</span>} | more<br /> rpm -q --changelog {<span style="color: #ff9900;">package-name</span>} | grep <span style="color: #996633;">CVE-NUMBER</span><br /> </code><br /> For example find out if CVE-2008-1927 has been applied to perl package or not, enter:<br /> <code># rpm -q --changelog perl|grep CVE-2008-1927</code><br /> Sample output:<br /><pre>- CVE-2008-1927 perl: double free on regular expressions with utf8 characters</pre>List all applied patches for php, enter:<br /> <code># rpm -q --changelog php</code><br /> OR<br /> <code># rpm -q --changelog php | more</code><br /> Sample output:<br /><div class="wp_syntax"><table><tbody><tr><td class="code"><pre class="txt" style="font-family: monospace;">* Tue Jun 03 2008 Joe Orton <jorton redhat.com=""> 5.1.6-20.el5_2.1<br />- add security fixes for CVE-2007-5898, CVE-2007-4782, CVE-2007-5899,<br /> CVE-2008-2051, CVE-2008-2107, CVE-2008-2108 (#445923)<br />&nbsp;<br />* Tue Jan 15 2008 Joe Orton <jorton redhat.com=""> 5.1.6-20.el5<br />- use magic.mime provided by file (#240845)<br />- fix possible crash with setlocale() (#428675)<br />&nbsp;<br />* Thu Jan 10 2008 Joe Orton <jorton redhat.com=""> 5.1.6-19.el5<br />- ext/date: fix test cases for recent timezone values (#266441)<br />&nbsp;<br />* Thu Jan 10 2008 Joe Orton <jorton redhat.com=""> 5.1.6-18.el5<br />- ext/date: updates for system tzdata support (#266441)<br />&nbsp;<br />* Wed Jan 09 2008 Joe Orton <jorton redhat.com=""> 5.1.6-17.el5<br />- ext/date: use system timezone database (#266441)<br />&nbsp;<br />* Tue Jan 08 2008 Joe Orton <jorton redhat.com=""> 5.1.6-16.el5<br />- add dbase extension in -common (#161639)<br />- add /usr/share/php to builtin include_path (#238455)<br />- ext/ldap: enable ldap_sasl_bind (#336221)<br />- ext/libxml: reset stream context (#298031)<br />.........<br />...<br />....<br />* Fri May 16 2003 Joe Orton <jorton redhat.com=""> 4.3.1-3<br />- link odbc module correctly<br />- patch so that php -n doesn't scan inidir<br />- run tests using php -n, avoid loading system modules<br />&nbsp;<br />* Wed May 14 2003 Joe Orton <jorton redhat.com=""> 4.3.1-2<br />- workaround broken parser produced by bison-1.875<br />&nbsp;<br />* Tue May 06 2003 Joe Orton <jorton redhat.com=""> 4.3.1-1<br />- update to 4.3.1; run test suite<br />- open extension modules with RTLD_NOW rather than _LAZY</jorton></jorton></jorton></jorton></jorton></jorton></jorton></jorton></jorton></pre></td></tr></tbody></table></div><h2>How do I find CVE for a rpm file itself?</h2>Above command will query installed package only. To query rpm file, enter:<br /> <code>$ rpm -qp --changelog rsnapshot-1.3.0-1.noarch.rpm | more</code><br /><h5>Further readings:</h5><ul><li>rpm command man page: <a href="http://www.manpager.com/redhat/man8/rpm.8.html" title="See rpm(8) redhat man page for more information and examples">rpm(8)</a></li></ul></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0tag:blogger.com,1999:blog-3950073089426506979.post-58858640138722443222016-07-10T00:22:00.002+02:002016-07-10T00:22:30.792+02:004 open source tools for Linux system monitoring<div dir="ltr" style="text-align: left;" trbidi="on">https://opensource.com/life/16/2/open-source-tools-system-monitoring<br /><br /><div class="panel-pane pane-entity-field pane-node-field-lead-image"> <div class="field field-name-field-lead-image field-type-image field-label-hidden"><div class="field-items"><div class="field-item even" rel="schema:contentUrl"><img alt="Linux system monitoring tools" class="image-full-size" height="292" src="https://opensource.com/sites/default/files/styles/image-full-size/public/images/life-uploads/htop.png?itok=6xXeIU67" title="Linux system monitoring tools" width="520" /></div></div></div></div><div class="panel-pane pane-entity-field pane-file-field-file-image-caption"> <div class="field field-name-field-file-image-caption field-type-text-long field-label-inline clearfix"><div class="field-label">Image by :&nbsp;</div><div class="field-items"><div class="field-item even">opensource.com<br /> </div></div></div></div><div class="panel-pane pane-block pane-service-links-service-links"> <div class="service-links"> <a class="service-links-reddit" href="http://reddit.com/submit?url=https%3A//opensource.com/life/16/2/open-source-tools-system-monitoring&amp;title=4%20open%20source%20tools%20for%20Linux%20system%20monitoring" rel="nofollow" title="Submit this post on reddit.com"></a> <span class="IN-widget" style="display: inline-block; line-height: 1; text-align: center; vertical-align: baseline;"><span style="display: inline-block ! important; font-size: 1px ! important; margin: 0px ! important; padding: 0px ! important; text-indent: 0px ! important; vertical-align: baseline ! important;"><span id="li_ui_li_gen_1468102863381_0"><a href="https://www.blogger.com/null" id="li_ui_li_gen_1468102863381_0-link"><span id="li_ui_li_gen_1468102863381_0-logo">in</span><span id="li_ui_li_gen_1468102863381_0-title"><span id="li_ui_li_gen_1468102863381_0-mark"></span><span id="li_ui_li_gen_1468102863381_0-title-text">Share</span></span></a></span></span><span style="display: inline-block ! important; font-size: 1px ! important; margin: 0px ! important; padding: 0px ! important; text-indent: 0px ! important; vertical-align: baseline ! important;"><span class="IN-right" id="li_ui_li_gen_1468102863393_1-container"><span class="IN-right" id="li_ui_li_gen_1468102863393_1"><span class="IN-right" id="li_ui_li_gen_1468102863393_1-inner"><span class="IN-right" id="li_ui_li_gen_1468102863393_1-content">84</span></span></span></span></span></span> </div></div><div class="panel-pane pane-entity-field pane-node-body"> <div class="field field-name-body field-type-text-with-summary field-label-hidden"><div class="field-items"><div class="field-item even">Information is the key to resolving any computer problem, including problems with or relating to Linux and the hardware on which it runs. There are many tools available for and included with most distributions even though they are not all installed by default. These tools can be used to obtain huge amounts of information.<br /> This article discusses some of the interactive command line interface (CLI) tools that are provided with or which can be easily installed on Red Hat related distributions including Red Hat Enterprise Linux, Fedora, CentOS, and other derivative distributions. Although there are GUI tools available and they offer good information, the CLI tools provide all of the same information and they are always usable because many servers do not have a GUI interface but all Linux systems have a command line interface.<br /> This article concentrates on the tools that I typically use. If I did not cover your favorite tool, please forgive me and let us all know what tools you use and why in the comments section.<br /> My go to tools for problem determination in a Linux environment are almost always the system monitoring tools. For me, these are top, atop, htop, and glances.<br /> All of these tools monitor CPU and memory usage, and most of them list information about running processes at the very least. Some monitor other aspects of a Linux system as well. All provide near real-time views of system activity.<br /> <h2>Load averages</h2>Before I go on to discuss the monitoring tools, it is important to discuss load averages in more detail.<br /> Load averages are an important criteria for measuring CPU usage, but what does this really mean when I say that the 1 (or 5 or 10) minute load average is 4.04, for example? Load average can be considered a measure of demand for the CPU; it is a number that represents the average number of instructions waiting for CPU time. So this is a true measure of CPU performance, unlike the standard "CPU percentage"&nbsp;which includes I/O wait times during which the CPU is not really working.<br /> For example, a fully utilized single processor system CPU would have a load average of 1. This means that the CPU is keeping up exactly with the demand; in other words it has perfect utilization. A load average of less than one means that the CPU is underutilized and a load average of greater than 1 means that the CPU is overutilized and that there is pent-up, unsatisfied demand. For example, a load average of 1.5 in a single CPU system indicates that one-third of the CPU instructions are forced to wait to be executed until the one preceding it has completed.<br /> This is also true for multiple processors. If a 4 CPU system has a load average of 4 then it has perfect utilization. If it has a load average of 3.24, for example, then three of its processors are fully utilized and one is utilized at about 76%. In the example above, a 4 CPU system has a 1 minute load average of 4.04 meaning that there is no remaining capacity among the 4 CPUs and a few instructions are forced to wait. A perfectly utilized 4 CPU system would show a load average of 4.00 so that the system in the example is fully loaded but not overloaded.<br /> The optimum condition for load average is for it to equal the total number of CPUs in a system. That would mean that every CPU is fully utilized and yet no instruction must be forced to wait. The longer-term load averages provide indication of the overall utilization trend.<br /> Linux Journal has <a href="http://www.linuxjournal.com/article/9001?page=0,0" target="_blank">an excellent article</a> describing load averages, the theory and the math behind them, and how to interpret them in the December 1, 2006 issue.<br /> <h2>Signals</h2>All of the monitors discussed here allow you to send <a href="https://en.wikipedia.org/wiki/Unix_signal" target="_blank">signals</a> to running processes. Each of these signals has a specific function though some of them can be defined by the receiving program using signal handlers.<br /> The separate <b>kill</b> command can also be used to send signals to processes outside of the monitors. The <b>kill -l</b> can be used to list all possible signals that can be sent. Three of these signals can be used to kill a process.<br /> <ul><li><b>SIGTERM (15):</b> Signal 15, SIGTERM is the default signal sent by top and the other monitors when the <b>k</b> key is pressed. It may also be the least effective because the program must have a signal handler built into it. The program's signal handler must intercept incoming signals and act accordingly. So for scripts, most of which do not have signal handlers, SIGTERM is ignored. The idea behind SIGTERM is that by simply telling the program that you want it to terminate itself, it will take advantage of that and clean up things like open files and then terminate itself in a controlled and nice manner.</li><li><b>SIGKILL (9):</b>&nbsp;Signal 9, SIGKILL provides a means of killing even the most recalcitrant programs, including scripts and other programs that have no signal handlers. For scripts and other programs with no signal handler, however, it not only kills the running script but it also kills the shell session in which the script is running; this may not be the behavior that you want. If you want to kill a process and you don't care about being nice, this is the signal you want. This signal cannot be intercepted by a signal handler in the program code.</li><li><b>SIGINT (2):</b> Signal 2, SIGINT can be used when SIGTERM does not work and you want the program to die a little more nicely, for example, without killing the shell session in which it is running. SIGINT sends an interrupt to the session in which the program is running. This is equivalent to terminating a running program, particularly a script, with the <b>Ctrl-C</b> key combination.</li></ul>To experiment with this, open a terminal session and create a file in /tmp named cpuHog and make it executable with the permissions rwxr_xr_x. Add the following content to the file.<br /> <pre>#!/bin/bash<br /># This little program is a cpu hog<br />X=0;while [ 1 ];do echo $X;X=$((X+1));done<br /></pre>Open another terminal session in a different window, position them adjacent to each other so you can watch the results and run <b>top</b> in the new session. Run the cpuHog program with the following command:<br /> <pre><b>/</b><b>tmp</b><b>/cpuHog</b><br /></pre>This program simply counts up by one and prints the current value of X to STDOUT. And it sucks up CPU cycles. The terminal session in which cpuHog is running should show a very high CPU usage in top. Observe the effect this has on system performance in top. CPU usage should immediately go way up and the load averages should also start to increase over time. If you want, you can open additional terminal sessions and start the cpuHog program in them so that you have multiple instances running.<br /> Determine the PID of the cpuHog program you want to kill. Press the <b>k</b> key and look at the message under the Swap line at the bottom of the summary section. Top asks for the PID of the process you want to kill. Enter that PID and press <strong>Enter</strong>. Now top asks for the signal number and displays the default of 15. Try each of the signals described here and observe the results.<br /> <h2>4 open source tools for Linux system monitoring</h2><h2>top</h2>One of the first tools I use when performing problem determination is <strong>top</strong>. I like it because it has been around <em>since forever</em> and is always available while the other tools may not be installed.<br /> The top program is a very powerful utility that provides a great deal of information about your running system. This includes data about memory usage, CPU loads, and a list of running processes including the amount of CPU time and memory being utilized by each process. Top displays system information in near real-time, updating (by&nbsp;default) every three seconds. Fractional seconds are allowed by top, although very small values can place a significant load the system. It is also interactive and the data columns to be displayed and the sort column can be modified.<br /> A sample output from the top program is shown in Figure 1 below. The output from top is divided into two sections which are called the "summary"&nbsp;section, which is the top section of the output, and the "process"&nbsp;section which is the lower portion of the output; I will use this terminology for top, atop, htop and glances in the interest of consistency.<br /> The top program has a number of useful interactive commands you can use to manage the display of data and to manipulate individual processes. Use the <b>h</b> command to view a brief help page for the various interactive commands. Be sure to press <b>h</b> twice to see both pages of the help. Use the <b>q</b> command to quit.<br /> <h3>Summary section</h3>The summary section of the output from top is an overview of the system status. The first line shows the system uptime and the 1, 5,&nbsp;and 15 minute load averages. In the example below, the load averages are 4.04, 4.17,&nbsp;and 4.06&nbsp;respectively.<br /> The second line shows the number of processes currently active and the status of each.<br /> The lines containing CPU statistics are shown next. There can be a single line which combines the statistics for all CPUs present in the system, as in the example below, or one line for each CPU; in the case of the computer used for the example, this is a single quad core CPU. Press the <strong>1</strong> key to toggle between the consolidated display of CPU usage and the display of the individual CPUs. The data in these lines is displayed as percentages of the total CPU time available.<br /> These and the other fields for CPU data are described below.<br /> <ul><li><strong>us: userspace –</strong> Applications and other programs running in user space, i.e., not in the kernel.</li><li><strong>sy: system calls –</strong> Kernel level functions. This does not include CPU time taken by the kernel itself, just the kernel system calls.</li><li><strong>ni: nice –</strong> Processes that are running at a positive nice level.</li><li><strong>id: idle –</strong> Idle time, i.e., time not used by any running process.</li><li><strong>wa: wait –</strong> CPU cycles that are spent waiting for I/O to occur. This is wasted CPU time.</li><li><strong>hi: hardware interrupts –</strong> CPU cycles that are spent dealing with hardware interrupts.</li><li><strong>si: software interrupts –</strong> CPU cycles spent dealing with software-created interrupts such as system calls.</li><li><strong>st: steal time –</strong> The percentage of CPU cycles that a virtual CPU waits for a real CPU while the hypervisor is servicing another virtual processor.</li></ul>The last two lines in the summary section are memory usage. They show the physical memory usage including both RAM and swap space.<br /> <div class="rtecenter"><img alt="Figure 1: The top command showing a fully utilized 4-core CPU." class="media-image attr__typeof__foaf:Image img__fid__310406 img__view_mode__default attr__format__default attr__field_file_image_alt_text[und][0][value]__Figure 1: The top command showing a fully utilized 4-core CPU. attr__field_file_image_title_text[und][0][value]__Figure attr__field_file_image_caption[und][0][value]__&lt;p&gt;Screenshot from David Both:&amp;nbsp;&lt;/p&gt; &lt;p&gt;Figure CPU.&lt;/p&gt; attr__field_file_image_caption[und][0][format]__panopoly_wysiwyg_text attr__field_folder[und]__9402" height="395" src="https://opensource.com/sites/default/files/resize/linuxsys_image1-520x395.jpg" style="height: 395px; width: 520px;" title="Figure 1: The top command showing a fully utilized 4-core CPU." width="520" /><br clear="left" /><sup>Figure 1: The top command showing a fully utilized 4-core CPU.</sup></div>You can use the <b>1</b> command to display CPU statistics as a single, global number as shown in Figure 1, above, or by individual CPU. The <b>l</b> command turns load averages on and off. The <b>t</b> and <b>m</b> commands rotate the process/CPU and memory lines of the summary section, respectively, through off, text only, and a couple types of bar graph formats.<br /> <h3>Process section</h3>The process section of the output from top is a listing of the running processes in the system—at least for the number of processes&nbsp;for which there is room on the terminal display. The default columns displayed by top are described below. Several other columns are available and each can usually be added with a single keystroke. Refer to the top man page for details.<br /> <ul><li><strong>PID – </strong>The Process ID.</li><li><strong>USER –</strong> The username of the process owner.</li><li><strong>PR –</strong> The priority of the process.</li><li><strong>NI –</strong> The nice number of the process.</li><li><strong>VIRT –</strong> The total amount of virtual memory allocated to the process.</li><li><strong>RES –</strong> Resident size (in kb unless otherwise noted) of non-swapped physical memory consumed by a process.</li><li><strong>SHR – </strong>The amount of shared memory in kb used by the process.</li><li><strong>S –</strong> The status of the process. This can be R for running, S for sleeping, and Z for zombie. Less frequently seen statuses can be T for traced or stopped, and D for uninterruptable sleep.</li><li><strong>%CPU – </strong>The percentage of CPU cycles, or time used by this process during the last measured time period.</li><li><strong>%MEM –</strong> The percentage of physical system memory used by the process.</li><li><strong>TIME+ –</strong> Total CPU time to 100ths of a second consumed by the process since the process was started.</li><li><strong>COMMAND – </strong>This is the command that was used to launch the process.</li></ul>Use the <b>Page Up</b> and <b>Page Down</b> keys to scroll through the list of running processes. The <b>d</b> or <b>s</b> commands are interchangeable and can be used to set the delay interval between updates. The default is three seconds,&nbsp;but I prefer a one second interval. Interval granularity can be as low as one-tenth (0.1) of a second but this will consume more of the CPU cycles you are trying to measure.<br /> You can use the <b>&lt;</b> and <b>&gt;</b> keys to sequence the sort column to the left or right.<br /> The <b>k</b> command is used to kill a process or the <b>r</b> command to renice it. You have to know the process ID (PID) of the process you want to kill or renice and that information is displayed in the process section of the top display. When killing a process, top asks first for the PID and then for the signal number to use in killing the process. Type them in and press the enter key after each. Start with signal 15, SIGTERM, and if that does not kill the process, use 9, SIGKILL.<br /> <h3>Configuration</h3>If you alter the top display, you can use the <b>W</b> (in uppercase) command to write the changes to the configuration file, ~/.toprc in your home directory.<br /> <hr /><h2>atop</h2>I also like atop. It is an excellent monitor to use when you need more details about that type of I/O activity. The default refresh interval is 10 seconds, but this can be changed using the interval <b>i</b>&nbsp;command to whatever is appropriate for what you are trying to do. atop cannot refresh at sub-second intervals like top can.<br /> Use the <b>h</b> command to display help. Be sure to notice that there are multiple pages of help and you can use the space bar to scroll down to see the rest.<br /> One nice feature of atop is that it can save raw performance data to a file and then play it back later for close inspection. This is handy for tracking down internmittent problems, especially ones that occur during times when you cannot directly monitor the system. The <b>atopsar</b> program is used to play back the data in the saved file.<br /> <div class="rtecenter"><img alt="Figure 2: The atop system monitor provides information about disk and network activity in addition to CPU and process data." class="media-image attr__typeof__foaf:Image img__fid__310411 img__view_mode__default attr__format__default attr__field_file_image_alt_text[und][0][value]__Figure 2: The atop system monitor provides information about disk and network activity in addition to CPU process data. attr__field_file_image_title_text[und][0][value]__Figure attr__field_file_image_caption[und][0][value]__&lt;p&gt;Screenshot from David Both.&amp;nbsp;&lt;/p&gt; &lt;p&gt;Figure data.&lt;/p&gt; attr__field_file_image_caption[und][0][format]__panopoly_wysiwyg_text attr__field_folder[und]__9402" height="314" src="https://opensource.com/sites/default/files/resize/linuxsys_image2-520x314.jpg" style="height: 314px; width: 520px;" title="Figure 2: The atop system monitor provides information about disk and network activity in addition to CPU and process data." width="520" /><sup>.</sup><br clear="left" /><sup>Figure 2: The atop system monitor provides information about disk and network activity in addition to CPU and process data.</sup></div><h3>Summary section</h3>atop contains much of the same information as top but also displays information about network, raw disk, and logical volume activity. Figure 2, above, shows these additional data in the columns at the top of the display. Note that if you have the horizontal screen real-estate to support a wider display, additional columns will be displayed. Conversely, if you have less horizontal width, fewer columns are displayed. I also like that atop displays the current CPU frequency and scaling factor—something I have not seen on any other of these monitors—on the second line in the rightmost two columns in Figure 2.<br /> <h3>Process section</h3>The atop process display includes some of the same columns as that for top, but it also includes disk I/O information and thread count for each process as well as virtual and real memory growth statistics for each process. As with the summary section, additional columns will display if there is sufficient horizontal screen real-estate. For example, in Figure 2, the RUID (Real User ID) of the process owner is displayed. Expanding the display will also show the EUID (Effective User ID) which might be important when programs run SUID (Set User ID).<br /> atop can also provide detailed information about disk, memory, network, and scheduling information for each process. Just press the <b>d</b>, <b>m</b>, <b>n</b> or <b>s</b> keys respectively to view that data. The <b>g</b> key returns the display to the generic process display.<br /> Sorting can be accomplished easily by using <b>C</b> to sort by CPU usage, <b>M</b> for memory usage, <b>D</b> for disk usage, <b>N</b> for network usage and <b>A</b> for automatic sorting. Automatic sorting usually sorts processes by the most busy resource. The network usage can only be sorted if the netatop kernel module is installed and loaded.<br /> You can use the <b>k</b> key to kill a process but there is no option to renice a process.<br /> By default, network and disk devices for which no activity occurs during a given time interval are not displayed. This can lead to mistaken assumptions about the hardware configuration of the host. The <b>f</b> command can be used to force atop to display the idle resources.<br /> <h3>Configuration</h3>The atop man page refers to global and user level configuration files, but none can be found in my own Fedora or CentOS installations. There is also no command to save a modified configuration and a save does not take place automatically when the program is terminated. So, there appears to be now way to make configuration changes permanent.<br /> <hr /><h2>htop</h2>The htop program is much like top <em>but on steroids</em>. It does look a lot like top, but it also provides some capabilities that top does not. Unlike atop, however, it does not provide any disk, network, or I/O information of any type.<br /> <div class="rtecenter"><br clear="left" /><img alt="Figure 3: htop has nice bar charts to to indicate resource usage and it can show the process tree." class="media-image attr__typeof__foaf:Image img__fid__310416 img__view_mode__default attr__format__default attr__field_file_image_alt_text[und][0][value]__Figure 3: htop has nice bar charts to indicate resource usage and it can show the process tree. attr__field_file_image_title_text[und][0][value]__Figure attr__field_file_image_caption[und][0][value]__&lt;p&gt;Screenshot from David Both.&amp;nbsp;&lt;/p&gt; &lt;p&gt;Figure tree.&lt;/p&gt; attr__field_file_image_caption[und][0][format]__panopoly_wysiwyg_text attr__field_folder[und]__9402" height="417" src="https://opensource.com/sites/default/files/resize/linuxsys_image3-520x417.jpg" style="height: 417px; width: 520px;" title="Figure 3: htop has nice bar charts to to indicate resource usage and it can show the process tree." width="520" /></div><div class="rtecenter"><sup>Figure 3: htop has nice bar charts to to indicate resource usage and it can show the process tree.</sup></div><h3><span style="line-height: 1.6;">Summary section</span></h3>The summary section of htop is displayed in two columns. It is very flexible and can be configured with several different types of information in pretty much any order you like. Although the CPU usage sections of top and atop can be toggled between a combined display and a display that shows one bar graph for each CPU, htop cannot. So it has a number of different options for the CPU display, including a single combined bar, a bar for each CPU, and various combinations in which specific CPUs can be grouped together into a single bar.<br /> I think this is a cleaner summary display than some of the other system monitors and it is easier to read. The drawback to this summary section is that some information is not available in htop that is available in the other monitors, such as CPU percentages by user, idle, and system time.<br /> The <b>F2</b> (Setup) key is used to configure the summary section of htop. A list of available data displays is shown and you can use function keys to add them to the left or right column and to move them up and down within the selected column.<br /> <h3>Process section</h3>The process section of htop is very similar to that of top. As with the other monitors, processes can be sorted any of several factors, including CPU or memory usage, user, or PID. Note that sorting is not possible when the tree view is selected.<br /> The <b>F6</b>&nbsp;key allows you to select the sort column; it displays a list of the columns available for sorting and you select the column you want and press the <strong>Enter</strong> key.<br /> You can use the up and down arrow keys to select a process. To kill a process, use the up and down arrow keys to select the target process and press the <b>k</b> key. A list of signals to send the process is displayed with 15, SIGTERM, selected. You can specify the signal to use, if different from SIGTERM. You could also use the <b>F7</b> and <b>F8</b> keys to renice the selected process.<br /> One command I especially like is <b>F5</b>&nbsp;which displays the running processes in a tree format making it easy to determine the parent/child relationships of running processes.<br /> <h3>Configuration</h3>Each user has their own configuration file, ~/.config/htop/htoprc and changes to the htop configuration are stored there automatically. There is no global configuration file for htop.<br /> <hr /><h2>glances</h2>I have just recently learned about glances, which can display more information about your computer than any of the other monitors I am currently familiar with. This includes disk and network I/O, thermal readouts that can display CPU and other hardware temperatures as well as fan speeds, and disk usage by hardware device and logical volume.<br /> The drawback to having all of this information is that glances uses a significant amount of CPU resurces itself. On my systems I find that it can use from about 10% to 18% of CPU cycles. That is a lot so you should consider that impact when you choose your monitor.<br /> <h3>Summary section</h3>The summary section of glances contains most of the same information as the summary sections of the other monitors. If you have enough horizontal screen real estate it can show CPU usage with both a bar graph and a numeric indicator, otherwise it will show only the number.<br /> <br /> <div class="rtecenter"><img alt="Figure 4: The glances interface with network, disk, filesystem, and sensor information." class="media-image attr__typeof__foaf:Image img__fid__310421 img__view_mode__default attr__format__default attr__field_file_image_alt_text[und][0][value]__Figure 4: The glances interface with network, disk, filesystem, and sensor information. attr__field_file_image_title_text[und][0][value]__Figure attr__field_file_image_caption[und][0][value]__&lt;p&gt;Screenshot from David Both.&amp;nbsp;&lt;/p&gt; &lt;p&gt;Figure information.&lt;/p&gt; attr__field_file_image_caption[und][0][format]__panopoly_wysiwyg_text attr__field_folder[und]__9402" height="441" src="https://opensource.com/sites/default/files/resize/linuxsys_image4-520x441.jpg" style="height: 441px; width: 520px;" title="Figure 4: The glances interface with network, disk, filesystem, and sensor information." width="520" /><br clear="left" /><sup>Figure 4: The glances interface with network, disk, filesystem, and sensor information.</sup></div>I like this summary section better than those of the other monitors; I think it provides the right information in an easily understandable format. As with atop and htop, you can press the <b>1</b> key to toggle between a display of the individual CPU cores or a global one with all of the CPU cores as a single average as shown in Figure 4, above.<br /> <h3>Process section</h3>The process section displays the standard information about each of the running processes. Processes can be sorted automatically <b>a</b>, or by CPU <b>c</b>, memory <b>m</b>, name <b>p</b>, user <b>u</b>, I/O rate <b>i</b>,&nbsp;or time <b>t</b>. When sorted automatically processes are first sorted by the most used resource.<br /> Glances also shows warnings and critical alerts at the very bottom of the screen, including the time and duration of the event. This can be helpful when attempting to diagnose problems when you cannot stare at the screen for hours at a time. These alert logs can be toggled on or off with the <b>l</b> command, warnings can be cleared with the <b>w</b> command while alerts and warnings can all be cleared with <b>x</b>.<br /> It is interesting that glances is the only one of these monitors that cannot be used to either kill or renice a process. It is intended strictly as a monitor. You can use the external <b>kill</b> and <b>renice</b> commands to manipulate processes.<br /> <h3>Sidebar</h3>Glances has a very nice sidebar that displays information that is not available in top or htop. Atop does display some of this data, but glances is the only monitor that displays the sensors data. Sometimes it is nice to see the temperatures inside your computer. The individual modules, disk, filesystem, network, and sensors can be toggled on and off using the <b>d</b>,<b>f</b>, <b>n</b>, and <b>s</b> commands, respectively. The entire sidebar can be toggled using <b>2</b>.<br /> Docker stats can be displayed with <b>D</b>.<br /> <h3>Configuration</h3>Glances does not require a configuration file to work properly. If you choose to have one, the system-wide instance of the configuration file would be located in /etc/glances/glances.conf. Individual users can have a local instance at ~/.config/glances/glances.conf which will override the global configuration. The primary purpose of these configuration files is to set thresholds for warnings and critical alerts. There is no way I can find to make other configuration changes—such as sidebar modules or the CPU displays—permanent. It appears that you must reconfigure those items every time you start glances.<br /> There is a document, /usr/share/doc/glances/glances-doc.html, that provides a great deal of information about using glances, and it explicitly states that you can use the configuration file to configure which modules are displayed. However, neither the information given nor the examples describe just how to do that.<br /> <hr /><h2>Conclusion</h2>Be sure to read the man pages for each of these monitors because there is a large amount of information about configuring and interacting with them. Also use the <strong>h</strong> key for help in interactive mode. This help can provide you with information about selecting and sorting the columns of data, setting the update interval and much more.<br /> These programs can tell you a great deal when you are looking for the cause of a problem. They can tell you when a process, and which one, is sucking up CPU time, whether there is enough free memory, whether processes are stalled while waiting for I/O such as disk or network access to complete, and much more.<br /> I strongly recommend that you spend time watching these monitoring programs while they run on a system that is functioning normally so you will be able to differentiate those things that may be abnormal while you are looking for the cause of a problem.<br /> You should also be aware that the act of using these monitoring tools alters the system's use of resources including memory and CPU time. top and most of these monitors use perhaps 2% or 3%&nbsp;of a system's CPU time. glances has much more impact than the others and can use between 10% and 20% of CPU time. Be sure to consider this when choosing your tools.<br /> I had originally intended to include SAR (System Activity Reporter) in this article but as this article grew longer it also became clear to me that SAR is significantly different from these monitoring tools and deserves to have a separate article. So with that in mind, I plan to write an article on SAR and the /proc filesystem, and a third article on how to use all of these tools to locate and resolve problems.<br /> </div></div></div></div></div>Sameh Attiahttp://www.blogger.com/profile/06829656663776752624noreply@blogger.com0