tag:code.tutsplus.com,2005:/categories/pythonEnvato Tuts+ Code - Python2018-11-08T10:22:31Ztag:code.tutsplus.com,2005:PostPresenter/cms-31840Create a Database Cluster in the Cloud With MongoDB Atlas<p>For years now, <a href="https://www.mongodb.com/?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" rel="external" target="_blank">MongoDB</a> has been the go-to NoSQL database for both individuals and enterprises building large-scale applications. It's open source, easily scalable, and provides high availability. It also supports very complex queries and fine-grained concurrency control. </p><p>However, necessary tasks such as installing the database, tuning it to maintain optimal performance over long periods of time, and securing it tend to require a lot of skilled and dedicated effort. </p><p>Fortunately, there's an easier alternative: <a href="https://www.mongodb.com/cloud/atlas?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" rel="external" target="_blank">MongoDB Atlas</a>, a fully managed, cloud version of the database.</p><p>With MongoDB Atlas, you can create a MongoDB cluster on any major cloud provider of your choice and start using that cluster in a matter of minutes. Using Atlas's browser-based user interface, you can also intuitively configure the cluster and monitor its performance.</p><p>In this tutorial, I'll show you how to create a MongoDB Atlas free tier cluster and use it in a Python application.</p><h2>Prerequisites</h2><p>Before you proceed, make sure you have the following installed and configured on your computer:</p><ul>
<li>
<a href="https://www.python.org/downloads/" rel="external" target="_blank">Python</a> 3.4 or higher</li>
<li>
<a href="https://pip.pypa.io/en/stable/installing/" rel="external" target="_blank">pip</a> 18.0 or higher</li>
</ul><h2>
<span class="sectionnum">1.</span> Creating a Cluster</h2><p>To be able to use MongoDB's cloud services, you'll need a MongoDB Atlas account. To create one, go to its <a href="https://www.mongodb.com/cloud/atlas?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" rel="external" target="_blank">home page</a> and press the <strong>Get started free</strong> button.</p><figure class="post_image"><img alt="Home page of MongoDB Atlas" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.09.18.jpg"></figure><p>After you complete the short sign-up form, you'll be redirected to the cluster creation wizard. In its first section, you'll have to pick the cloud provider and region you prefer.<br></p><p>To minimize network latency, you'd ideally pick a region that's nearest to your computer. For now, however, because we are creating a free tier cluster, make sure the region you select is one that has a free tier available. Additionally, if you are using a Google Cloud VM or an Amazon EC2 instance as your development environment, do select the corresponding cloud provider first.</p><figure class="post_image"><img alt="Cloud provider and region selection" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.15.05.png"></figure><p>In the <strong>Cluster Tier</strong> section, select the <strong>M0</strong> option to create your free tier cluster. It offers 512 MB of storage space, a recent version of MongoDB with WiredTiger as the storage engine, a replica set of three nodes, and a generous 10 GB of bandwidth per week.</p><figure class="post_image"><img alt="Cluster tier selection" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.15.53.png"></figure><p>Lastly, give a meaningful name to the cluster and press the <strong>Create Cluster</strong> button.</p><figure class="post_image"><img alt="Cluster name form" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.19.06.png"></figure><p>MongoDB Atlas will now take about five minutes to set up your cluster.</p><h2>
<span class="sectionnum">2.</span> Configuring the Cluster</h2><p>Before you start using the cluster, you'll have to provide a few security-related details, so switch to the <strong>Security</strong> tab.</p><p>First, in the <strong>MongoDB Users</strong> section, you must create a new user for yourself by pressing the <strong>Add new user</strong> button. In the dialog that pops up, type in your desired username and password, select the <strong>Read and write to any database</strong> privilege, and press the <strong>Add User</strong> button.</p><figure class="post_image"><img alt="Add new user form" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.28.16.png"></figure><p>Next, in the <strong>IP Whitelist</strong> section, you must provide a list of IP addresses from which you'll be accessing the cluster. For now, providing the current IP address of your computer is sufficient.</p><p>Press the <strong>Add IP address</strong> button to create a new IP address entry. In the dialog that pops up, press the <strong>Add current IP address</strong> button to autofill the <strong>Whitelist Entry</strong> field. Additionally, if you don't have a static IP address, it's a good idea to mark it is a temporary entry by checking the <strong>Save as temporary whitelist</strong> option. Finally, press <strong>Confirm</strong> to add the entry.</p><figure class="post_image"><img alt="Add whitelist entry form" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.31.27.png"></figure><h2>
<span class="sectionnum">3.</span> Getting the Connection String</h2><p>You'll need a valid connection string to connect to your cluster from your application. To get it, go to the <strong>Overview</strong> tab and press the <strong>Connect</strong> button.</p><p>In the dialog that opens, select the <strong>Connect Your Application</strong> option and press the <strong>I'm using driver 3.6 or later</strong> button. You should now be able to see your connection string. It won't have your actual password, so you'll have to put it in manually. After you do so, make a note of the string so you can use it later.</p><figure class="post_image"><img alt="Connecting your application dialog" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screen%20Shot%202018-09-09%20at%2008.44.08.png"></figure><h2>
<span class="sectionnum">4.</span> Installing the Python Driver</h2><p>To be able to interact with your MongoDB Atlas cluster programmatically, you must have a <a href="https://docs.mongodb.com/ecosystem/drivers/#drivers?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">MongoDB driver</a> installed on your computer. For the Python programming language, <a href="https://github.com/mongodb/mongo-python-driver" rel="external" target="_blank">PyMongo</a> is the most popular driver available today. The recommended way to install it on your computer is to use the <code class="inline">pip</code> module as shown below:</p><pre class="brush: bash noskimlinks noskimwords">python3 -m pip install pymongo --user</pre><p>You may have noticed that your MongoDB Atlas connection string is a <code class="inline">mongodb+srv://</code> URI. To enable the driver to work with DNS SRV records, you must also install the <code class="inline">dnspython</code> module. Here's how:</p><pre class="brush: bash noskimlinks noskimwords">python3 -m pip install dnspython --user</pre><h2>
<span class="sectionnum">5.</span> Connecting to the Cluster</h2><p>You can now use your MongoDB cluster from any Python application. To follow along with me, create a new Python script and open it using any code editor.</p><p>Inside the script, to be able to interact with the cluster, you'll need an instance of the <code class="inline">MongoClient</code> class. As the only argument to its constructor, pass your connection string.</p><pre class="brush: python noskimlinks noskimwords">import pymongo
my_client = pymongo.MongoClient(
'mongodb+srv://alice:myPassword@mylittlecluster-qsart.gcp.mongodb.net/test?retryWrites=true'
)</pre><p>The above constructor returns immediately and will not raise any errors. Therefore, to check if you've successfully established a connection, I suggest you try to perform an operation on the cluster. A call to the <code class="inline">server_info()</code> method, which gets you various details about your MongoDB instance, will suffice.</p><p>If there are no errors in your connection string, the call to the <code class="inline">server_info()</code> method will succeed. However, if the username or password you specified is incorrect, you'll encounter an <code class="inline">OperationFailure</code> error. The following code shows you how to catch it:</p><pre class="brush: python noskimlinks noskimwords">try:
print("MongoDB version is %s" %
my_client.server_info()['version'])
except pymongo.errors.OperationFailure as error:
print(error)
quit(1)</pre><p>You can now go ahead and try running your script.</p><figure class="post_image"><img alt="Python script output" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/31840/image/Screenshot%20from%202018-09-12%2001:51:33.png"></figure><h2>
<span class="sectionnum">6.</span> Inserting Documents</h2><p>The default connection string you get from MongoDB Atlas's web interface mentions a database named <code class="inline">test</code>. Let's continue to use the same database. Here's how you can get a reference to it:</p><pre class="brush: python noskimlinks noskimwords">my_database = my_client.test</pre><p>A MongoDB database is composed of one or more collections, which are nothing but groups of <a href="https://docs.mongodb.com/manual/reference/bson-types/index.html?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">BSON documents (short for binary JSON)</a>. Your free tier cluster on MongoDB Atlas can have a maximum of 500 collections.</p><p>For the sake of a realistic example, let's create a new collection named <code class="inline">foods</code>. With PyMongo, you don't have to explicitly call any method to do so. You can simply reference it as if it exists already.</p><pre class="brush: python noskimlinks noskimwords">my_collection = my_database.foods</pre><p>It's worth mentioning that the above code doesn't create the <code class="inline">foods</code> collection immediately. It's created only after you add a document to it. So let's now create and add a new document containing nutritional data about a food item.</p><p>Using the <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_one?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">insert_one()</a></code> method is the simplest way to add a document to a collection. To specify the contents of the document, you pass a Python dictionary to the method. The following sample code shows you how:</p><pre class="brush: python noskimlinks noskimwords">my_collection.insert_one({
"_id": 1,
"name": "pizza",
"calories": 266,
"fats": {
"saturated": 4.5,
"trans": 0.2
},
"protein": 11
})</pre><p>Adding documents one at a time can be inefficient. With the <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_many?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">insert_many()</a></code> method, you can add several documents to your collection at once. It expects an array of dictionaries as its input. The following code adds two more documents to the collection:</p><pre class="brush: python noskimlinks noskimwords">my_collection.insert_many([
{
"_id": 2,
"name": "hamburger",
"calories": 295, "protein": 17,
"fats": { "saturated": 5.0, "trans": 0.8 },
},
{
"_id": 3,
"name": "taco",
"calories": 226, "protein": 9,
"fats": { "saturated": 4.4, "trans": 0.5 },
}
])</pre><p>The <code class="inline">_id</code> field you see in the above documents is a field that's used as a primary key by MongoDB. As such, it is immutable and must be present in all MongoDB documents. If you forget to include it while creating your document, PyMongo will add it for you automatically and assign an auto-generated unique value to it.</p><h2>
<span class="sectionnum">7.</span> Running Queries</h2><p>When you've added a few documents to your collection, you can run queries on it by calling the <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.find?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">find()</a></code> method, which returns a <code class="inline">Cursor</code> object you can iterate over. If you don't pass any arguments to it, <code class="inline">find</code> returns all the documents in the collection.</p><p>The following code shows you how to print the names of all the food items present in our collection:</p><pre class="brush: python noskimlinks noskimwords">my_cursor = my_collection.find()
for item in my_cursor:
print(item["name"])
# Output is:
# pizza
# hamburger
# taco</pre><p>If you want the <code class="inline">find()</code> method to return only those documents that match specific criteria, you must pass a Python dictionary to it. For example, if you want to find the document whose <code class="inline">name</code> field is set to "pizza", you could use the following code:</p><pre class="brush: python noskimlinks noskimwords">my_cursor = my_collection.find({
"name": "pizza"
})</pre><p>For more complex queries, you can use MongoDB's <a href="https://docs.mongodb.com/manual/reference/operator/query/?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">intuitively named query operators</a> in the dictionaries you pass to the <code class="inline">find()</code> method. For instance, the following code shows you how to use the <code class="inline">$lt</code> operator to find documents whose <code class="inline">calories</code> field is set to a value that's less than 280:</p><pre class="brush: python noskimlinks noskimwords">my_cursor = my_collection.find({
"calories": { "$lt": 280 }
})
for item in my_cursor:
print("Name: %s, Calories: %d" %
(item["name"], item["calories"]))
# Output is:
# Name: pizza, Calories: 266
# Name: taco, Calories: 226</pre><p>By using the dot notation, you can also use nested fields in your queries. The following code shows you how to find documents whose <code class="inline">trans</code> field, which is inside the <code class="inline">fats</code> field, is set to a value that's greater than or equal to 0.5:</p><pre class="brush: python noskimlinks noskimwords">my_cursor = my_collection.find({
"fats.trans": { "$gte": 0.5 }
})
for item in my_cursor:
print("Name: %s, Trans fats: %.2f" %
(item["name"], item["fats"]["trans"]))
# Output is:
# Name: hamburger, Trans fats: 0.80
# Name: taco, Trans fats: 0.50</pre><h2>
<span class="sectionnum">8.</span> Updating and Deleting Documents</h2><p>Very similar to the <code class="inline">insert_one()</code> and <code class="inline">insert_many()</code> methods are the <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_one?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">update_one()</a></code> and <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_many?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">update_many()</a></code> methods, which you can use to change the contents of documents that are already inside your collection. Both the update methods, in addition to new data, need a query to zero in on the documents that need to be changed.</p><p>You can use a variety of update operators in your update methods. The most commonly used one is <code class="inline">$set</code>, which lets you add new fields or update the values of existing fields. The following code shows you how to add two new fields named <code class="inline">fiber</code> and <code class="inline">sugar</code> to the document whose <code class="inline">name</code> field is set to "taco":</p><pre class="brush: python noskimlinks noskimwords">my_collection.update_one(
{ "name": "taco" }, # query
{
"$set": { # new data
"fiber": 3.95,
"sugar": 0.9
}
}
)</pre><p>If the query you pass to the <code class="inline">update_one()</code> method returns more than one document, only the first document is updated. The <code class="inline">update_many()</code> method doesn't have this limitation.</p><p>Lastly, by using the <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_one?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">delete_one()</a></code> and <code class="inline"><a href="http://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_many?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" target="_self">delete_many()</a></code> methods, you can delete documents in your collections. Both the methods need a query to determine which documents need to be deleted. Here's how you can delete all documents whose <code class="inline">calories</code> field is set to a value that's less than 300:</p><pre class="brush: python noskimlinks noskimwords">my_collection.delete_many({
"calories": {
"$lt": 300
}
})
# Deletes all the three documents</pre><h2>Conclusion</h2><p>If you are a web or mobile application developer who wants to use MongoDB for storing data, the MongoDB Atlas service is for you. It lets you focus on developing your application instead of worrying about details such as security, performance, and adherence to best practices. In this tutorial, you learned how to create a MongoDB cluster using the service, connect to it, and perform basic read and write operations on it.</p><p>To learn more about MongoDB Atlas, you can refer to the official <a href="https://docs.atlas.mongodb.com/?jmp=tuts&amp;Americas_USA_tuts_stitch_tutorials_ATLAS" rel="external" target="_blank">documentation</a>.</p>2018-11-08T10:22:31.085Z2018-11-08T10:22:31.085ZAshraff Hathibelagaltag:code.tutsplus.com,2005:PostPresenter/cms-31648New Course: Machine Learning With Google TensorFlow<p>Do you want to learn how to use deep neural networks and machine learning in your apps? Try our new course, <a href="https://code.tutsplus.com/courses/learn-machine-learning-with-google-tensorflow?ec_promo=short_course" target="_self">Learn Machine Learning With Google TensorFlow</a>, to get up-to-date with the latest developments in this cutting-edge technology.</p><h2>What You’ll Learn</h2><p>More and more users are starting to expect their apps and devices to be capable of recognizing patterns and making decisions in much the same way as humans do.</p><p>With TensorFlow—a fast, powerful, and open-source machine learning framework developed by Google—working with deep learning algorithms is easier than you may think. </p><p>In this short course, Envato Tuts+ instructor <a href="https://tutsplus.com/authors/ashraff-hathibelagal?ec_promo=short_course" target="_self">Ashraff Hathibelagal</a> shows you how to create, train, and evaluate a variety of deep neural networks using TensorFlow's high-level Keras API.<br></p><figure class="post_image"><a href="https://code.tutsplus.com/courses/learn-machine-learning-with-google-tensorflow?ec_promo=short_course" target="_self"><img alt="Training a convolutional network" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/31648/image/Training-and-Using-a-Convolutional-Network.jpg"></a></figure><p>Want to find out more about the course and what it covers? Check out the quick intro video below.</p><h2>Watch the Introduction</h2><figure><script src="//fast.wistia.com/assets/external/E-v1.js" async="async"></script><div class="wistia_responsive_padding"><div class="wistia_responsive_wrapper"><div class="wistia_embed wistia_async_d871dywcxy videoFoam=true"> </div></div></div></figure><h2>Take the Course</h2><p>You can take our new course straight away with a <a href="https://tutsplus.com/subscribe-with-elements?ec_promo=short_course" target="_self">subscription to Envato Elements</a>. For a single low monthly fee, you get access not only to this course, but also to our growing library of over 1,000 video courses and industry-leading eBooks on Envato Tuts+. </p><p>Plus you now get unlimited downloads from the huge Envato Elements library of 650,000+ creative assets. Create with unique fonts, photos, graphics and templates, and deliver better projects faster.</p>2018-08-14T09:47:58.000Z2018-08-14T09:47:58.000ZAndrew Blackmantag:code.tutsplus.com,2005:PostPresenter/cms-30555Getting Started With the Fabric Python Library<p>Fabric is a Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks. Fabric is very simple and powerful and can help to automate repetitive command-line tasks. This approach can save time by automating your entire workflow. </p><p>This tutorial will cover how to use Fabric to integrate with SSH and automate tasks.</p><h2>Installation</h2><p>Fabric is best installed via pip:</p>
<pre class="brush: bash noskimlinks noskimwords">$ pip install fabric
</pre><h2>Getting Started With Fabric</h2><h3>Usage</h3><p>Below is a simple function demonstrating how to use Fabric.</p><pre class="brush: python noskimlinks noskimwords">def welcome():
print("Welcome to getting started with Fabric!")</pre><p>The program above is then saved as <code class="inline">fabfile.py</code> in your current working directory. The welcome function can be executed with the <code class="inline">fab</code> tool as follows:</p><pre class="brush: bash noskimlinks noskimwords">$ fab welcome
Welcome to getting started with Fabric</pre><p>Fabric provides the fab command which reads its configuration from a file, <code class="inline">fabfile.py</code>. The file should be in the directory from which the command is run. A standard fabfile contains the functions to be executed on a remote host or a group of remote hosts.</p><h2>Features</h2><p>Fabric implements functions which can be used to communicate with remote hosts:</p><h3><strong>fabric.operations.run()</strong></h3><p>This operation is used to run a shell command on a remote host. <br></p><p><strong>Examples</strong></p><pre class="brush: bash noskimlinks noskimwords">run("ls /var/www/")
run("ls /home/userx", shell=False)
output = run('ls /var/www/mysites'</pre><h3>
<strong></strong>fabric.operations.get()</h3><p>This function is used to download file(s) from a remote host. The example below shows how to download a backup from a remote server.</p><pre class="brush: plain noskimlinks noskimwords"># Downloading a back-up
get("/backup/db.bak", "./db.bak")
</pre><h3>fabric.operations.put()</h3><p>This functions uploads file(s) to a remote host. For example:</p><pre class="brush: bash noskimlinks noskimwords">with cd('/tmp'):
put('/path/to/local/test.txt', 'files')</pre><h3>fabric.operations.reboot()</h3><p>As the name suggests, this function reboots a system server.</p><pre class="brush: bash noskimlinks noskimwords"># Reboot the remote system
reboot()
</pre><h3>fabric.operations.sudo()</h3><p>This function is used to execute commands on a remote host with superuser privileges. Additionally, you can also pass an additional user argument which allows you to run commands as another user other than root.</p><p><strong>Example</strong></p><pre class="brush: plain noskimlinks noskimwords"># Create a directory
sudo("mkdir /var/www")</pre><h3>fabric.operations.local()</h3><p>This function is used to run a command on the local system. An example is:<br></p><pre class="brush: plain noskimlinks noskimwords"># Extract the contents of a tar archive
local("tar xzvf /tmp/trunk/app.tar.gz")
# Remove a file
local("rm /tmp/trunk/app.tar.gz")</pre><h3>fabric.operations.prompt()</h3><p>The function prompts the user with text and returns the input.</p><p><strong>Examples</strong></p><pre class="brush: bash noskimlinks noskimwords"># Simplest form:
environment = prompt('Please specify target environment: ')
# specify host
env_host = prompt('Please specify host:')
</pre><h3>fabric.operations.require()</h3><p>This function is used to check for given keys in a shared environment dict. If not found, the operation is aborted.<br></p><h2>SSH Integration</h2><p>One of the ways developers interact with remote servers besides FTP clients is through SSH. SSH is used to connect to remote servers and do everything from basic configuration to running Git or initiating a web server.</p><p>With Fabric, you can perform SSH activities from your local computer.<br></p><p>The example below defines functions that show how to check free disk space and host type. It also defines which host will run the command:</p><pre class="brush: python noskimlinks noskimwords"># Import Fabric's API module
from fabric.api import run
env.hosts = '159.89.39.54'
# Set the username
env.user = "root"
def host_type():
run('uname -s')
def diskspace():
run('df')
def check():
# check host type
host_type()
# Check diskspace
diskspace()
</pre><p>In order to run this code, you will need to run the following command on the terminal:</p><pre class="brush: bash noskimlinks noskimwords">fab check</pre><h4>Output <br>
</h4><pre class="brush: bash noskimlinks noskimwords">fab check[159.89.39.54] Executing task 'check'
[159.89.39.54] run: uname -s
[159.89.39.54] Login password for 'root':
[159.89.39.54] out: Linux
[159.89.39.54] out:
[159.89.39.54] run: df
[159.89.39.54] out: Filesystem 1K-blocks Used Available Use% Mounted on
[159.89.39.54] out: udev 242936 0 242936 0% /dev
[159.89.39.54] out: tmpfs 50004 6020 43984 13% /run
[159.89.39.54] out: /dev/vda1 20145768 4398716 15730668 22% /
[159.89.39.54] out: tmpfs 250012 1004 249008 1% /dev/shm
[159.89.39.54] out: tmpfs 5120 0 5120 0% /run/lock
[159.89.39.54] out: tmpfs 250012 0 250012 0% /sys/fs/cgroup
[159.89.39.54] out: /dev/vda15 106858 3426 103433 4% /boot/efi
[159.89.39.54] out: tmpfs 50004 0 50004 0% /run/user/0
[159.89.39.54] out: none 20145768 4398716 15730668 22% /var/lib/docker/aufs/mnt/781d1ce30963c0fa8af93b5679bf96425a0a10039d10be8e745e1a22a9909105
[159.89.39.54] out: shm 65536 0 65536 0% /var/lib/docker/containers/036b6bcd5344f13fdb1fc738752a0850219c7364b1a3386182fead0dd8b7460b/shm
[159.89.39.54] out: none 20145768 4398716 15730668 22% /var/lib/docker/aufs/mnt/17934c0fe3ba83e54291c1aebb267a2762ce9de9f70303a65b12f808444dee80
[159.89.39.54] out: shm 65536 0 65536 0% /var/lib/docker/containers/fd90146ad4bcc0407fced5e5fbcede5cdd3cff3e96ae951a88f0779ec9c2e42d/shm
[159.89.39.54] out: none 20145768 4398716 15730668 22% /var/lib/docker/aufs/mnt/ba628f525b9f959664980a73d94826907b7df31d54c69554992b3758f4ea2473
[159.89.39.54] out: shm 65536 0 65536 0% /var/lib/docker/containers/dbf34128cafb1a1ee975f56eb7637b1da0bfd3648e64973e8187ec1838e0ea44/shm
[159.89.39.54] out:
Done.
Disconnecting from 159.89.39.54... done.
</pre><h2>Automating Tasks</h2><p>Fabric enables you to run commands on a remote server without needing to log in to the remote server.</p><p>Remote execution with Fabric can lead to security threats since it requires an open SSH port, especially on Linux machines. <br></p><p>For instance, let's assume you want to update the system libraries on your remote server. You don't necessarily need to execute the tasks every other time. You can just write a simple fab file which you will run every time you want to execute the tasks.</p><p>In this case, you will first import the Fabric API's module:</p><pre class="brush: python noskimlinks noskimwords">from fabric.api import *</pre><p>Define the remote host you want to update:</p><pre class="brush: python noskimlinks noskimwords">env.hosts = '159.89.39.54'</pre><p>Set the username of the remote host:</p><pre class="brush: python noskimlinks noskimwords">env.user = "root"</pre><p>Although it's not recommended, you might need to specify the password to the remote host.</p><p>Lastly, define the function that updates the libraries in your remote host.</p><pre class="brush: python noskimlinks noskimwords">def update_libs():
"""
Update the default OS installation's
basic default tools.
"""
run("apt-get update")</pre><p>Now that your fab file is ready, all you need to do is execute it as follows:</p><pre class="brush: bash noskimlinks noskimwords">$ fab update</pre><p>You should see the following result:</p><pre class="brush: bash noskimlinks noskimwords">$ fab update
[159.89.39.54] Executing task 'update'
[159.89.39.54] run: apt-get update
[159.89.39.54] Login password for 'root':</pre><p>If you didn't define the password, you will be prompted for it.</p><p>After the program has finished executing the defined commands, you will get the following response, if no errors occur:</p><pre class="brush: bash noskimlinks noskimwords">$ fab update
............
Disconnecting from 159.89.39.54... done. </pre><h2>Conclusion</h2><p>This tutorial has covered what is necessary to get started with Fabric locally and on remote hosts. You can now confidently start writing your own scripts for building, monitoring or maintaining remote servers.</p>2018-06-15T13:31:32.000Z2018-06-15T13:31:32.000ZEsther Vaatitag:code.tutsplus.com,2005:PostPresenter/cms-31199New Course: Connect a Database to Your Python Application<p>Python is a great way to create web apps, but what happens when you need to add a database? You'll find out exactly how to handle that in our new short course, <a href="https://code.tutsplus.com/courses/connect-a-database-to-your-python-application?ec_promo=short_course" target="_self">Connect a Database to Your Python Application</a>.</p><h2>What You’ll Learn</h2><p>In this short course, <a href="https://tutsplus.com/authors/derek-jensen?ec_promo=short_course" target="_self">Derek Jensen</a> will show you how easy it is to integrate a simple database into an existing Python application. </p><figure class="post_image"><a href="https://code.tutsplus.com/courses/connect-a-database-to-your-python-application?ec_promo=short_course" target="_self"><img alt="Database in a Python app" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/31199/image/Python-db.jpg"></a></figure><p>In this course, you'll be using SQLite, but you'll learn how to implement your database in a way that makes it easy to change or upgrade later. You'll learn how to use the Repository pattern to make your app more maintainable, testable, and flexible.<br></p><h2>Watch the Introduction</h2><figure><script src="//fast.wistia.com/assets/external/E-v1.js" async="async"></script><div class="wistia_responsive_padding"><div class="wistia_responsive_wrapper"><div class="wistia_embed wistia_async_pfvl6sunvb videoFoam=true"> </div></div></div></figure><h2>Take the Course</h2><p>You can take our new course straight away with a <a href="https://tutsplus.com/subscribe-with-elements?ec_promo=short_course" target="_self">subscription to Envato Elements</a>. For a single low monthly fee, you get access not only to this course, but also to our growing library of over 1,000 video courses and industry-leading eBooks on Envato Tuts+. </p><p>Plus you now get unlimited downloads from the huge Envato Elements library of 580,000+ creative assets. Create with unique fonts, photos, graphics and templates, and deliver better projects faster.</p>2018-05-29T09:27:38.000Z2018-05-29T09:27:38.000ZAndrew Blackmantag:code.tutsplus.com,2005:PostPresenter/cms-30623Introduction to Machine Learning in Python<p>Machine learning is the act of giving computers the ability to learn without explicitly programming them. This is done by giving data to computers and having them transform the data into decision models which are then used for future predictions.</p><p>In this tutorial, we will talk about machine learning and some of the fundamental concepts that are required in order to get started with machine learning. We will also devise a few Python examples to predict certain elements or events.</p><h2>Introduction to Machine Learning</h2><p>Machine learning is a type of technology that aims to learn from experience. For example, as a human, you can learn how to play chess simply by observing other people playing chess. In the same way, computers are programmed by providing them with data from which they learn and are then able to predict future elements or conditions.</p><p>Let's say, for instance, that you want to write a program that can tell whether a certain type of fruit is an orange or a lemon. You might find it easy to write such a program and it will give the required results, but you might also find that the program doesn't work effectively for large datasets. This is where machine learning comes into play.</p><p>There are various steps involved in machine learning:</p><ol>
<li>collection of data</li>
<li>filtering of data</li>
<li>analysis of data</li>
<li>algorithm training</li>
<li>testing of the algorithm</li>
<li>using the algorithm for future predictions</li>
</ol><p>Machine learning uses different kinds of algorithms to find patterns, and these algorithms are classified into two groups:</p><ul>
<li>supervised learning</li>
<li>unsupervised learning</li>
</ul><h3>Supervised Learning</h3><p>Supervised learning is the science of training a computer to recognize elements by giving it sample data. The computer then learns from it and is able to predict future datasets based on the learned data.</p><p>For example, you can train a computer to filter out spam messages based on past information.<br></p><p>Supervised learning has been used in many applications, e.g. Facebook, to search images based on a certain description. You can now search images on Facebook with words that describe the contents of the photo. Since the social networking site already has a database of captioned images, it is able to search and match the description to features from photos with some degree of accuracy.</p><p>There are only two steps involved in supervised learning:</p><ul>
<li>training <br>
</li>
<li>testing</li>
</ul><p>Some of the supervised learning algorithms include:</p><ul>
<li>decision trees</li>
<li>support vector machines</li>
<li>naive Bayes</li>
<li>k-nearest neighbor</li>
<li>linear regression<br>
</li>
</ul><h4>Example</h4><p>We are going to write a simple program to demonstrate how supervised learning works using the Sklearn library and the Python language. Sklearn is a machine learning library for the Python programming language with a range of features such as multiple analysis, regression, and clustering algorithms.</p><p>Sklearn also interoperates well with the <a href="https://code.tutsplus.com/tutorials/introducing-numpy--cms-28703" rel="external" target="_blank">NumPy</a> and SciPy libraries.</p><h3>Install Sklearn</h3><p>The Sklearn installation guide offers a very simple way of installing it for multiple platforms. It requires several dependencies:</p><ul>
<li>Python (&gt;= 2.7 or &gt;= 3.3),</li>
<li>NumPy (&gt;=1.82)<br>
</li>
<li>SciPy (&gt;=0.13.3)<br>
</li>
</ul><p>If you already have these dependencies, you can install Sklearn as simply as:</p><pre class="brush: bash noskimlinks noskimwords">pip install -U scikit-learn</pre><p>An easier way is to simply install Anaconda. This takes care of all the dependencies so you don't have to worry about installing them one by one.</p><p>To test if Sklearn is running properly, simply import it from a Python interpreter as follows:</p><pre class="brush: plain noskimlinks noskimwords">import sklearn</pre><p>If no error occurs, then you are good to go.</p><p>Now that we are done with the installation, let's get back to our problem. We want to able to differentiate between different animals. So we will design an algorithm that can tell specifically whether a given animal is either a horse or a chicken.</p><p>We first need to collect some sample data from each type of animal. Some sample data is shown in the table below.</p><table>
<thead><tr>
<th>Height (inches)</th>
<th>Weight (kg)<br>
</th>
<th>Temperature (Celsius)<br>
</th>
<th>Label<br>
</th>
</tr></thead>
<tbody>
<tr>
<td>7<br>
</td>
<td>0.6<br>
</td>
<td>40</td>
<td>Chicken (0)<br>
</td>
</tr>
<tr>
<td>7<br>
</td>
<td>0.6</td>
<td>41<br>
</td>
<td>Chicken (0)<br>
</td>
</tr>
<tr>
<td>37<br>
</td>
<td>0.8</td>
<td>37<br>
</td>
<td>Horse (1)</td>
</tr>
<tr>
<td>37<br>
</td>
<td>0.8</td>
<td>38<br>
</td>
<td>Horse (1)</td>
</tr>
</tbody>
</table><p>The sample data we have obtained gives some of the common features of the two animals and data from two of the animals. The larger the sample data, the more accurate and less biased the results will be.</p><p>With this type of data, we can code an algorithm and train it to recognize an animal based on the trained values and classify it either as a horse or a chicken. Now we will go ahead and write the algorithm that will get the job done.</p><p>First, import the tree module from Sklearn.</p><pre class="brush: python noskimlinks noskimwords">from sklearn import tree</pre><p>Define the features you want to use to classify the animals.</p><pre class="brush: python noskimlinks noskimwords">features = [[7, 0.6, 40], [7, 0.6, 41], [37, 600, 37], [37, 600, 38]]</pre> <p>Define the output each classifier will give. A chicken will be represented by 0, while a horse will be represented by 1.</p><pre class="brush: python noskimlinks noskimwords">#labels = [chicken, chicken, horse, horse]
# we use 0 to represent a chicken and 1 to represent a horse
labels = [0, 0, 1, 1]
</pre><p>We then define the classifier which will be based on a decision tree.</p><pre class="brush: python noskimlinks noskimwords">classifier = tree.DecisionTreeClassifier()</pre><p>Feed or fit your data to the classifier.</p><pre class="brush: python noskimlinks noskimwords">classifier.fit(features, labels)
</pre><p>The complete code for the algorithm is shown below.</p><pre class="brush: python noskimlinks noskimwords">from sklearn import tree
features = [[7, 0.6, 40], [7, 0.6, 41], [37, 600, 37], [37, 600, 38]]
#labels = [chicken, chicken, horse, horse]
labels = [0, 0, 1, 1]
classif = tree.DecisionTreeClassifier()
classif.fit(features, labels)</pre><p>We can now predict a given set of data. Here's how to predict an animal with a height of 7 inches, a weight of 0.6 kg, and a temperature of 41:</p><pre class="brush: python noskimlinks noskimwords">from sklearn import tree
features = [[7, 0.6, 40], [7, 0.6, 41], [37, 600, 37], [37, 600, 38]]
#labels = [chicken, chicken, horse, horse]
labels = [0, 0, 1, 1]
classif = tree.DecisionTreeClassifier()
classif.fit(features, labels)
print classif.predict([[7, 0.6, 41]])
#output
# [0] or a Chicken</pre><p>Here's how to predict an animal with a height of 38 inches, a weight of 600 kg, and a temperature of 37.5:</p><pre class="brush: python noskimlinks noskimwords">from sklearn import tree
features = [[7, 0.6, 40], [7, 0.6, 41], [37, 600, 37], [37, 600, 38]]
#labels = [chicken, chicken, horse, horse]
labels = [0, 0, 1, 1]
classif = tree.DecisionTreeClassifier()
classif.fit(features, labels)
print classif.predict([[38, 600, 37.5]])
# output
# [1] or a Horse
</pre><p>As you can see above, you have trained the algorithm to learn all the features and names of the animals, and the knowledge of this data is used for testing new animals.</p><h2>Unsupervised Learning</h2><p>Unsupervised learning is when you train your machine with only a set of inputs. The machine will then be able to find a relationship between the input data and any other you might want to predict. Unlike in supervised learning, where you present a machine with some data to train on, unsupervised learning is meant to make the computer find patterns or relationships between different datasets.</p><p>Unsupervised learning can be further subdivided into:</p><ul>
<li>clustering</li>
<li>association<br>
</li>
</ul><p><strong>Clustering</strong>: Clustering means grouping data inherently. For example, you can classify the shopping habits of consumers and use it for advertising by targeting the consumers based on their purchases and shopping habits.</p><p><strong>Association</strong>: Association is where you identify rules that describe large sets of your data. This type of learning can be applicable in associating books based on author or category, whether motivational, fictional, or educational books.</p><p>Some of the popular unsupervised learning algorithms include:</p><ul>
<li>k-means clustering</li>
<li>hierarchical clustering</li>
</ul><p>Unsupervised learning will be an important technology in the near future. This is due to the fact that there is a lot of unfiltered data which has not yet been digitized.</p><h2>Conclusion</h2><p>I hope this tutorial has helped you get started with machine learning. This is just an introduction—machine learning has a lot to cover, and this is just a fraction of what machine learning can do.</p><p>Additionally, don’t hesitate to see what we have available for sale and for study <a href="https://codecanyon.net/search/python" target="_self">in the Envato Market</a>, and don't hesitate to ask any questions and provide your valuable feedback using the feed below.</p><p>Your decision to use either a supervised or unsupervised machine learning algorithm will depend on various factors, such as the structure and size of the data.<br></p><p>Machine learning can be applied in almost all areas of our lives, e.g. in fraud prevention, personalizing news feed in social media sites to fit users' preferences, email and malware filtering, weather predictions, and even in the e-commerce sector to predict consumer shopping habits.</p>2018-05-16T12:31:06.000Z2018-05-16T12:31:06.000ZEsther Vaatitag:code.tutsplus.com,2005:PostPresenter/cms-30957How to Use TensorFlow Mobile in Android Apps<p>With <a href="https://www.tensorflow.org/" rel="external" target="_blank">TensorFlow</a>, one of the most popular machine learning frameworks available today, you can easily create and train deep models—also commonly referred to as deep feed-forward neural networks—that can solve a variety of complex problems, such as image classification, object detection, and natural language comprehension. <a href="https://www.tensorflow.org/mobile/" rel="external" target="_blank">TensorFlow Mobile</a> is a library designed to help you leverage those models in your mobile apps.</p><p>In this tutorial, I'll show you how to use TensorFlow Mobile in Android Studio projects.<br></p><h2>Prerequisites<br>
</h2><p>To be able to follow this tutorial, you'll need:<br></p><ul>
<li>
<a href="https://developer.android.com/studio/index.html" rel="external" target="_blank">Android Studio</a> 3.0 or higher</li>
<li>
<a href="https://www.tensorflow.org/install/" rel="external" target="_blank">TensorFlow</a> 1.5.0 or higher</li>
<li>an Android device running API level 21 or higher</li>
<li>and a basic understanding of the TensorFlow framework</li>
</ul><h2>
<span class="sectionnum">1.</span> Creating a Model</h2><p>Before we start using TensorFlow Mobile, we'll need a trained TensorFlow model. Let's create one now.<br></p><p>Our model is going to be very basic. It will behave like an XOR gate, taking two inputs, both of which can be either zero or one, and producing one output, which will be zero if both the inputs are identical and one otherwise. Additionally, because it's going to be a deep model, it will have two hidden layers, one with four neurons, and another with three neurons. You are free to change the number of hidden layers and the numbers of neurons they contain.<br></p><p>In order to keep this tutorial short, instead of using the low-level TensorFlow APIs directly, we'll be using <a href="https://github.com/tflearn/tflearn" rel="external" target="_blank">TFLearn</a>, a popular wrapper framework for TensorFlow offering more intuitive and concise APIs. If you don't have it already, use the following command to install it inside your TensorFlow virtual environment:</p><pre class="brush: bash noskimlinks noskimwords">pip install tflearn</pre><p>To start creating the model, create a Python script named <strong>create_model.py</strong>, preferably in an empty directory, and open it with your favorite text editor.</p><p>Inside the file, the first thing we need to do is import the TFLearn APIs.</p><pre class="brush: python noskimlinks noskimwords">import tflearn</pre><p>Next, we must create the training data. For our simple model, there will be only four possible inputs and outputs, which will resemble the contents of the XOR gate's truth table.</p><pre class="brush: python noskimlinks noskimwords">X = [
[0, 0],
[0, 1],
[1, 0],
[1, 1]
]
Y = [
[0], # Desired output for inputs 0, 0
[1], # Desired output for inputs 0, 1
[1], # Desired output for inputs 1, 0
[0] # Desired output for inputs 1, 1
]</pre><p>It is usually a good idea to use random values picked from a uniform distribution while assigning initial weights to all the neurons in the hidden layers. To generate the values, use the <code class="inline">uniform()</code> method.</p><pre class="brush: python noskimlinks noskimwords">weights = tflearn.initializations.uniform(minval = -1, maxval = 1)</pre><p>At this point, we can start creating the layers of our neural network. To create the input layer, we must use the <code class="inline">input_data()</code> method, which allows us to specify the number of inputs the network can accept. Once the input layer is ready, we can call the <code class="inline">fully_connected()</code> method multiple times to add more layers to the network.</p><pre class="brush: python noskimlinks noskimwords"># Input layer
net = tflearn.input_data(
shape = [None, 2],
name = 'my_input'
)
# Hidden layers
net = tflearn.fully_connected(net, 4,
activation = 'sigmoid',
weights_init = weights
)
net = tflearn.fully_connected(net, 3,
activation = 'sigmoid',
weights_init = weights
)
# Output layer
net = tflearn.fully_connected(net, 1,
activation = 'sigmoid',
weights_init = weights,
name = 'my_output'
)</pre><p>Note that in the above code, we have given meaningful names to the input and output layers. Doing so is important because we'll need them while using the network from our Android app. Also note that the hidden and output layers are using the <code class="inline">sigmoid</code> activation function. You are free to experiment with other activation functions, such as <code class="inline">softmax</code>, <code class="inline">tanh</code>, and <code class="inline">relu</code>.</p><p>As the last layer of our network, we must create a regression layer using the <code class="inline">regression()</code> function, which expects a few hyper-parameters as its arguments, such as the network's learning rate and the optimizer and loss functions it should use. The following code shows you how to use stochastic gradient descent, SGD for short, as the optimizer function and mean square as the loss function:</p><pre class="brush: python noskimlinks noskimwords">net = tflearn.regression(net,
learning_rate = 2,
optimizer = 'sgd',
loss = 'mean_square'
)</pre><p>Next, in order to let the TFLearn framework know that our network model is actually a deep neural network model, we must call the <code class="inline">DNN()</code> function.</p><pre class="brush: python noskimlinks noskimwords">model = tflearn.DNN(net)</pre><p>The model is now ready. All we need to do now is train it using the training data we created earlier. So call the <code class="inline">fit()</code> method of the model and, along with the training data, specify the number of training epochs to run. Because the training data is very small, our model will need thousands of epochs to attain reasonable accuracy.</p><pre class="brush: python noskimlinks noskimwords">model.fit(X, Y, 5000)</pre><p>Once the training is complete, we can call the <code class="inline">predict()</code> method of the model to check if it is generating the desired outputs. The following code shows you how to check the outputs for all valid inputs:</p><pre class="brush: python noskimlinks noskimwords">print("1 XOR 0 = %f" % model.predict([[1,0]]).item(0))
print("1 XOR 1 = %f" % model.predict([[1,1]]).item(0))
print("0 XOR 1 = %f" % model.predict([[0,1]]).item(0))
print("0 XOR 0 = %f" % model.predict([[0,0]]).item(0))</pre><p>If you run the Python script now, you should see output that looks like this:</p><figure class="post_image"><img alt="Predictions after training" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot%20from%202018-04-12%2007-11-00.png" title="Image: https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot from 2018-04-12 07-11-00.png"></figure><p>Note that the outputs are never exactly 0 or 1. Instead, they are floating-point numbers that are either close to zero or close to one. Therefore, while using the outputs, you might want to use Python's <code class="inline">round()</code> function.</p><p>Unless we explicitly save the model after training it, we will lose it as soon as the script ends. Fortunately, with TFLearn, a simple call to the <code class="inline">save()</code> method saves the model. However, to be able to use the saved model with TensorFlow Mobile, before saving it, we must make sure we remove all the training-related operations, which are present in the <code class="inline">tf.GraphKeys.TRAIN_OPS</code> collection, associated with it. The following code shows you how to do so:</p><pre class="brush: python noskimlinks noskimwords"># Remove train ops
with net.graph.as_default():
del tf.get_collection_ref(tf.GraphKeys.TRAIN_OPS)[:]
# Save the model
model.save('xor.tflearn')</pre><p>If you run the script again, you'll see that it generates a checkpoint file, a metadata file, an index file, and a data file, all of which when used together can quickly recreate our trained model.</p><h2>
<span class="sectionnum">2.</span> Freezing the Model</h2><p>In addition to saving the model, we must freeze it before we can use it with TensorFlow Mobile. The process of freezing a model, as you might have guessed, involves converting all its variables into constants. Additionally, a frozen model must be a single binary file that conforms to the Google Protocol Buffers serialization format.</p><p>Create a new Python script named <strong>freeze_model.py</strong> and open it using a text editor. We'll be writing all the code to freeze our model inside this file.</p><p>Because TFLearn doesn't have any functions for freezing models, we'll have to use the TensorFlow APIs directly now. Import them by adding the following line to the file:</p><pre class="brush: python noskimlinks noskimwords">import tensorflow as tf</pre><p>Throughout the script, we'll be using a single TensorFlow session. To create the session, use the constructor of the <code class="inline">Session</code> class.</p><pre class="brush: python noskimlinks noskimwords">with tf.Session() as session:
# Rest of the code goes here</pre><p>At this point, we must create a <code class="inline">Saver</code> object by calling the <code class="inline">import_meta_graph()</code> function and passing the name of the model's metadata file to it. In addition to returning a <code class="inline">Saver</code> object, the <code class="inline">import_meta_graph()</code> function also automatically adds the graph definition of the model to the graph definition of the session.</p><p>Once the saver is created, we can initialize all the variables that are present in the graph definition by calling the <code class="inline">restore()</code> method, which expects the path of the directory containing the model's latest checkpoint file.<br></p><pre class="brush: python noskimlinks noskimwords">my_saver = tf.train.import_meta_graph('xor.tflearn.meta')
my_saver.restore(session, tf.train.latest_checkpoint('.'))</pre><p>At this point, we can call the <code class="inline">convert_variables_to_constants()</code> function to create a frozen graph definition where all the variables of the model are replaced with constants. As its inputs, the function expects the current session, the current session's graph definition, and a list containing the names of the model's output layers.</p><pre class="brush: python noskimlinks noskimwords">frozen_graph = tf.graph_util.convert_variables_to_constants(
session,
session.graph_def,
['my_output/Sigmoid']
)</pre><p>Calling the <code class="inline">SerializeToString()</code> method of the frozen graph definition gives us a binary protobuf representation of the model. By using Python's basic file I/O facilities, I suggest you save it as a file named <strong>frozen_model.pb</strong>.</p><pre class="brush: python noskimlinks noskimwords">with open('frozen_model.pb', 'wb') as f:
f.write(frozen_graph.SerializeToString())</pre><p>You can run the script now to generate the frozen model.</p><p>We now have everything we need to start using TensorFlow Mobile.</p><h2>
<span class="sectionnum">3.</span> Android Studio Project Setup</h2><p>The TensorFlow Mobile library is available on JCenter, so we can directly add it as an <code class="inline">implementation</code> dependency in the <code class="inline">app</code> module's <strong>build.gradle</strong> file.</p><pre class="brush: groovy noskimlinks noskimwords">implementation 'org.tensorflow:tensorflow-android:1.7.0'</pre><p>To add the frozen model to the project, place the <strong>frozen_model.pb</strong> file in the project's <strong>assets</strong> folder.</p><h2>
<span class="sectionnum">4.</span> Initializing the TensorFlow Interface</h2><p>TensorFlow Mobile offers a simple interface we can use to interact with our frozen model. To create the interface, use the constructor of the <code class="inline">TensorFlowInferenceInterface</code> class, which expects an <code class="inline">AssetManager</code> instance and the filename of the frozen model.</p><pre class="brush: actionscript3 noskimlinks noskimwords">thread {
val tfInterface = TensorFlowInferenceInterface(assets,
"frozen_model.pb")
// More code here
}</pre><p>In the above code, you can see that we're spawning a new thread. Doing so, although not always necessary, is recommended in order to make sure that the app's UI stays responsive.</p><p>To be sure that TensorFlow Mobile has managed to read our model's file correctly, let's now try printing the names of all the operations that are present in the model's graph. To get a reference to the graph, we can use the <code class="inline">graph()</code> method of the interface, and to get all the operations, the <code class="inline">operations()</code> method of the graph. The following code shows you how:</p><pre class="brush: actionscript3 noskimlinks noskimwords">val graph = tfInterface.graph()
graph.operations().forEach {
println(it.name())
}</pre><p>If you run the app now, you should be able to see over a dozen operation names printed in Android Studio's <strong>Logcat</strong> window. Among all those names, if there were no errors while freezing the model, you'll be able to find the names of the input and output layers: <strong>my_input/X</strong> and <strong>my_output/Sigmoid</strong>.</p><figure class="post_image"><img alt="Logcat window showing list of operations" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot%20from%202018-04-12%2007-19-01.png" title="Image: https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot from 2018-04-12 07-19-01.png"></figure><h2>
<span class="sectionnum">5.</span> Using the Model</h2><p>To make predictions with the model, we must put data into its input layer and retrieve data from its output layer. To put data into the input layer, use the <code class="inline">feed()</code> method of the interface, which expects the name of the layer, an array containing the inputs, and the dimensions of the array. The following code shows you how to send the numbers <code class="inline">0</code> and <code class="inline">1</code> to the input layer:</p><pre class="brush: actionscript3 noskimlinks noskimwords">tfInterface.feed("my_input/X",
floatArrayOf(0f, 1f), 1, 2)</pre><p>After loading data into the input layer, we must run an inference operation using the <code class="inline">run()</code> method, which expects the name of the output layer. Once the operation is complete, the output layer will contain the prediction of the model. To load the prediction into a Kotlin array, we can use the <code class="inline">fetch()</code> method. The following code shows you how to do so:</p><pre class="brush: actionscript3 noskimlinks noskimwords">tfInterface.run(arrayOf("my_output/Sigmoid"))
val output = floatArrayOf(-1f)
tfInterface.fetch("my_output/Sigmoid", output)</pre><p>How you use the prediction is of course up to you. For now, I suggest you simply print it.</p><pre class="brush: actionscript3 noskimlinks noskimwords">println("Output is ${output[0]}")</pre><p>You can run the app now to see that the model's prediction is correct.</p><figure class="post_image"><img alt="Logcat window displaying the prediction" data-src="https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot%20from%202018-04-12%2007-20-12.png" title="Image: https://cms-assets.tutsplus.com/uploads/users/362/posts/30957/image/Screenshot from 2018-04-12 07-20-12.png"></figure><p>Feel free to change the numbers you feed to the input layer to confirm that the model's predictions are always correct.</p><h2>Conclusion</h2><p>You now know how to create a simple TensorFlow model and use it with TensorFlow Mobile in Android apps. You don't always have to limit yourself to your own models, though. With the skills you learned today, you should have no problems using larger models, such as MobileNet and Inception, available in the TensorFlow <a href="https://github.com/tensorflow/models" rel="external" target="_blank" title="Link: https://github.com/tensorflow/models">model zoo</a>. Note, however, that such models will lead to larger APKs, which may create issues for users with low-end devices.</p><p>To learn more about TensorFlow Mobile, do refer to the <a href="https://www.tensorflow.org/mobile/mobile_intro" rel="external" target="_blank" title="Link: https://www.tensorflow.org/mobile/mobile_intro">official documentation</a>.</p>2018-04-30T17:00:00.000Z2018-04-30T17:00:00.000ZAshraff Hathibelagaltag:code.tutsplus.com,2005:PostPresenter/cms-30459Introduction to Network Programming in Python<p>This tutorial will give an introduction to sockets in Python and how to use the socket module to build HTTP servers and clients in Python. It will also cover Tornado, a Python networking library which is ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.</p>
<h2>What Are Sockets?</h2>
<p>A socket is a link between two applications that can communicate with one another (either locally on a single machine or remotely between two machines in separate locations).</p><p>Basically, sockets act as a communication link between two entities, i.e. a server and a client. A server will give out information being requested by a client. For example, when you visited this page, the browser created a socket and connected to the server. <br></p><h3>The socket Module</h3><p>In order to create a socket, you use the <code class="inline">socket.socket()</code> function, and the syntax is as simple as:</p><pre class="brush: python noskimlinks noskimwords">import socket
s= socket.socket (socket_family, socket_type, protocol=0)</pre><p>Here is the description of the arguments:</p><ul>
<li>
<strong>socket_family</strong>: Represents the address (and protocol) family. It can be either AF_UNIX or AF_INET.</li>
<li>
<strong>socket_type</strong>: Represents the socket type, and can be either SOCK_STREAM or SOCK_DGRAM.</li>
<li>
<strong>protocol</strong>: This is an optional argument, and it usually defaults to 0.</li>
</ul><p>After obtaining your socket object, you can then create a server or client as desired using the methods available in the socket module.</p><h2>Create a Simple Client</h2><p>Before we get started, let's look at the client socket methods available in Python.</p><ul>
<li>
<code class="inline">s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)</code><br>
</li>
<li>
<b><code class="inline">s.connect()</code></b>: Initiates a TCP server connection.</li>
</ul><p>To create a new socket, you first import the socket method of the socket class.</p><pre class="brush: python noskimlinks noskimwords">import socket</pre><p>Next, we'll create a stream (TCP) socket as follows:</p><pre class="brush: python noskimlinks noskimwords">stream_socket = socket.socket( socket.AF_INET, socket.SOCK_STREAM )</pre><p>The <code class="inline">AF_INET</code> argument indicates that you're requesting an Internet Protocol (IP) socket, specifically IPv4. The second argument is the transport protocol type <code class="inline">SOCK_STREAM</code> for TCP sockets. Additionally, you can also create an IPv6 socket by specifying the socket <code class="inline">AF_INET6</code> argument.</p><p>Specify the server.<br></p><pre class="brush: python noskimlinks noskimwords">server = "localhost"</pre><p>Specify the port we want to communicate with.</p><pre class="brush: python noskimlinks noskimwords">port =80</pre><p>Connect the socket to the port where the server is listening.</p><pre class="brush: python noskimlinks noskimwords">server_address = ((host, port))
stream_socket.connect(server_address)
</pre><p>It's important to note that the host and port must be a tuple.</p><p>Send a data request to the server:</p><pre class="brush: python noskimlinks noskimwords">message = 'message'
stream_socket.sendall(message)</pre><p>Get the response from the server:</p><pre class="brush: python noskimlinks noskimwords">data = sock.recv(10)
print data</pre><p>To close a connected socket, you use the close method:</p><pre class="brush: python noskimlinks noskimwords">stream_socket.close()</pre><p>Below is the full code for the Client/Server.</p><pre class="brush: python noskimlinks noskimwords">import socket
import sys
# Create a TCP/IP socket
stream_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Define host
host = 'localhost'
# define the communication port
port = 8080
# Connect the socket to the port where the server is listening
server_address = ((host, port))
print "connecting"
stream_socket.connect(server_address)
# Send data
message = 'message'
stream_socket.sendall(message)
# response
data = stream_socket.recv(10)
print data
print 'socket closed'
stream_socket.close()
</pre><h2>Build a Simple Server</h2><p>Now let's take a look at a simple Python server. The following are the socket server methods available in Python.</p><ul>
<li><code class="inline">s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)</code></li>
<li>
<code class="inline">s.bind()</code>: Binds address (hostname, port number) to socket.</li>
<li>
<code class="inline">s.listen()</code>: Sets up and starts TCP listener.</li>
<li>
<code class="inline">s.accept()</code>: Accepts TCP client connection.<br>
</li>
</ul><p>We will follow the following steps: <br></p><ul>
<li>Create a socket.</li>
<li>Bind the socket to a port.<br>
</li>
<li>Start accepting connections on the socket.</li>
</ul><p>Here is the server program.</p><pre class="brush: python noskimlinks noskimwords">import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Define host
host = 'localhost'
# define the communication port
port = 8080
# Bind the socket to the port
sock.bind((host, port))
# Listen for incoming connections
sock.listen(1)
# Wait for a connection
print 'waiting for a connection'
connection, client = sock.accept()
print client, 'connected'
# Receive the data in small chunks and retransmit it
data = connection.recv(16)
print 'received "%s"' % data
if data:
connection.sendall(data)
else:
print 'no data from', client
# Close the connection
connection.close()
</pre><p>The server is now ready for incoming connections.</p><p>Now run the client and server programs in separate terminal windows, so they can communicate with each other.</p><h4>
<strong></strong>Server Output</h4><pre class="brush: bash noskimlinks noskimwords">$ python server.py
waiting for a connection
('127.0.0.1', 47050) connected
received "message"
</pre><h4>Client Output</h4><pre class="brush: bash noskimlinks noskimwords">$ python client.py
connecting
message
socket closed
</pre><h2>The Tornado Framework</h2><p>The Tornado framework is one of the libraries available for network programming in Python. In this section, we will discuss this library and show how to use it to build WebSockets.</p><p>Tornado is a Python web framework and asynchronous networking library. Tornado uses the non-blocking network I/O, and hence is capable of scaling to tens of thousands of open connections. This trait makes it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.</p><p>Let's create a simple Tornado WebSocket:</p><pre class="brush: python noskimlinks noskimwords">import tornado.ioloop
import tornado.web
class ApplicationHandler(tornado.web.RequestHandler):
def get(self):
self.message = message = """&lt;html&gt;
&lt;head&gt;
&lt;title&gt;Tornado Framework&lt;/title&gt;
&lt;/head&gt;
&lt;body
&lt;h2&gt;Welcome to the Tornado framework&lt;/h2&gt;
&lt;/body&gt;
&lt;/html&gt;"""
self.write(message)
if __name__ == "__main__":
application = tornado.web.Application([
(r"/", ApplicationHandler),
])
application.listen(5001)
tornado.ioloop.IOLoop.instance().start()
</pre><p>In the code above:</p><ul>
<li>We define the class <code class="inline">ApplicationHandler</code> which serves as the handler for a request and returns a response using the <code class="inline">write()</code> method.</li>
<li>The <code class="inline">main</code> method is the entry for the program.</li>
<li>
<code class="inline">tornado.web.Application</code> creates a base for the web application and takes a collection of handlers, i.e. ApplicationHandler.</li>
<li>The Application listens on port 5000, and a client can communicate to this application using the same port.</li>
<li>
<code class="inline">tornado.ioloop.IOLoop.instance().start()</code> creates a nonblocking thread for an application.</li>
</ul><p>If we run the application, we will get the result as shown in the screenshot below.</p><figure class="post_image"><img alt="The results of running our application" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30459/image/imgonline-com-ua-resize-WPU7bHxRxZ2.jpg" title="Image: https://cms-assets.tutsplus.com/uploads/users/1885/posts/30459/image/imgonline-com-ua-resize-WPU7bHxRxZ2.jpg"></figure><h2>Conclusion</h2><p>By now you must have grasped the basics of socket programming in Python and how you can build a simple server and client. Feel free to experiment by building your own chat client. For more information, visit the official <a href="https://docs.python.org/2/howto/sockets.html" rel="external" target="_blank">Python docs.</a></p><p>Additionally, don’t hesitate to see what we have available for sale and for study <a href="https://codecanyon.net/search/python" title="Link: http://codecanyon.net/category/python" target="_self">in the Envato Market</a>, and don't hesitate to ask any questions and provide your valuable feedback using the feed below.</p>2018-04-25T12:00:00.000Z2018-04-25T12:00:00.000ZEsther Vaatitag:code.tutsplus.com,2005:PostPresenter/cms-30994New eBooks Available for Subscribers in April 2018<p>Do you want to learn more about Android development with Kotlin? How about learning concurrency in Python, or creating high-performing websites with TypeScript? Our <a href="https://tutsplus.com/ebooks?ec_promo=teaser_post_ebooks" target="_self">latest batch of eBooks</a> will teach you all you need to know about these topics and more.</p><figure class="post_image"><a href="https://code.tutsplus.com/ebooks?ec_promo=teaser_post_ebooks" target="_self"><img alt="New Code eBooks for Envato Elements subscribers" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/ebook-promo-apr18.jpg"></a></figure><h2>What You’ll Learn<br>
</h2><p>This month we’ve made seven new eBooks available for Envato Tuts+ subscribers to download. Here’s a summary of what you can learn from them.</p><h3>1. <a href="https://code.tutsplus.com/ebooks/android-development-with-kotlin?ec_promo=teaser_post_ebooks" target="_self">Android Development With Kotlin</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/android-development-with-kotlin?ec_promo=teaser_post_ebooks" target="_self"><img alt="Android Development with Kotlin" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/Kotlin.jpg"></a></figure><p>Kotlin is a language that helps developers build amazing Android applications easily and effectively. This book demonstrates how common examples that are typical for Android development can be simplified using Kotlin. It also shows all the benefits, improvements and new possibilities provided by this language.</p><p>Throughout the book, you will learn how to use Kotlin for Android development. You will get to know and understand the most important Kotlin features and how they can be used. By the end, you will be ready to start your own adventure with Android development with Kotlin.</p><h3>2. <a href="https://code.tutsplus.com/ebooks/react-native-building-mobile-apps-with-javascript?ec_promo=teaser_post_ebooks" target="_self">React Native: Building Mobile Apps With JavaScript</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/react-native-building-mobile-apps-with-javascript?ec_promo=teaser_post_ebooks" target="_self"><img alt="React Native Building Mobile Apps With JavaScript" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/React-Native.jpg"></a></figure><p>The emergence of React Native has made creating mobile apps in JavaScript easier for developers. This book introduces you to the React Native framework and the mobile app development process. </p><p>It starts with how React Native fits into the world of hybrid apps, and why it’s a popular framework. You’ll learn how React Native works under the hood, compiling JavaScript to Native code to bridge JavaScript and native apps. You'll also learn how to write React Native components, use the ReactJS way of structuring your app, and much more.</p><h3>3. <a href="https://code.tutsplus.com/ebooks/full-stack-web-development-with-raspberry-pi-3?ec_promo=teaser_post_ebooks" target="_self">Full-Stack Web Development With Raspberry Pi 3</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/full-stack-web-development-with-raspberry-pi-3?ec_promo=teaser_post_ebooks" target="_self"><img alt="Full Stack Web Development with Raspberry Pi 3" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/Raspberry-Pi-3.jpg"></a></figure><p>The Raspberry Pi is a very popular portable computer for running full-stack web applications. This book will empower you to master this rapidly evolving technology to develop complex web applications and interfaces.</p><p>The book starts by familiarizing you with the various components that make up the web development stack and that will integrate into your Raspberry Pi-powered web applications. It also introduces the Raspberry Pi computer and teaches you how to get up and running with a brand new one, before taking you through the full process of creating complex full-stack web applications on the Raspberry Pi 3.</p><h3>4. <a href="https://code.tutsplus.com/ebooks/learning-redux?ec_promo=teaser_post_ebooks" target="_self">Learning Redux</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/learning-redux?ec_promo=teaser_post_ebooks" target="_self"><img alt="Learning Redux" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/Redux.jpg"></a></figure><p>The book starts with a short introduction to the principles and the ecosystem of Redux, and then moves on to show you how to implement the basic elements of Redux and integrate Redux with other frameworks, such as React and Angular. In the main section of the book, you are going to develop a blog application from scratch using Redux. </p><p>By the end, you'll be able to develop and maintain Redux applications with ease. In addition to learning about Redux, you are going be familiar with its ecosystem, and you'll learn a lot about JavaScript itself, including best practices and patterns.<br></p><h3>5. <a href="https://code.tutsplus.com/ebooks/learning-concurrency-in-python?ec_promo=teaser_post_ebooks" target="_self">Learning Concurrency in Python</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/learning-concurrency-in-python?ec_promo=teaser_post_ebooks" target="_self"><img alt="Learning Concurrency in Python" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/Concurrency.jpg"></a></figure><p>This book introduces you to some of the most popular Python libraries and frameworks and goes in-depth into how you can leverage these libraries for your own concurrent, high-performing Python programs. We'll cover the fundamental concepts of concurrency needed to be able to write your own concurrent and parallel software systems in Python.<br></p><p>The book will guide you down the path to mastering Python concurrency, giving you all the necessary hardware and theoretical knowledge. It covers concepts such as debugging and exception handling as well as some of the most popular libraries and frameworks that allow you to create event-driven and reactive systems.</p><p>By the end of the book, you'll have learned the techniques to write incredibly efficient concurrent systems that follow best practices.</p><h3>6. <a href="https://code.tutsplus.com/ebooks/typescript-high-performance?ec_promo=teaser_post_ebooks" target="_self">TypeScript High Performance</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/typescript-high-performance?ec_promo=teaser_post_ebooks" target="_self"><img alt="TypeScript High Performance" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/TypeScript.jpg"></a></figure><p>In a world where a tiny decrease in frames per second has a big impact on customer engagement, writing highly scalable code is more of a necessity than a luxury. Using TypeScript, you get type checking during development, giving you the power to write optimized code quickly. </p><p>The book starts by explaining the efficient implementation of basic data structures, data types, and flow control. You will then learn efficient use of advanced language constructs and asynchronous programming. Further, you'll learn different configurations available with TSLint to improve code quality and performance. You'll also learn about profiling, building, deploying, and more.</p><h3>7. <a href="https://code.tutsplus.com/ebooks/javascript-by-example?ec_promo=teaser_post_ebooks" target="_self">JavaScript by Example</a>
</h3><figure class="post_image"><a href="https://code.tutsplus.com/ebooks/javascript-by-example?ec_promo=teaser_post_ebooks" target="_self"><img alt="JavaScript by Example" data-src="https://cms-assets.tutsplus.com/uploads/users/48/posts/30994/image/JavaScript-by-Example.jpg"></a></figure><p>JavaScript is the programming language that all web developers need to learn. This book teaches you JavaScript through practical projects like building a to-do list app, working with images and text to build a Meme creator, developing a responsive Event Registration form, building a weather application, and more. </p><p>Towards the end of the book, you will meet React, Facebook's JavaScript library for building user interfaces. You'll throw together a blog with React, and get a feel for why this kind of JavaScript framework is used to build large-scale applications. To make your blog more maintainable and scalable, you'll use Redux to manage data across React components.<br></p><h2>Start Reading With a Combined Subscription</h2><p>You can read our new eBooks straight away with a <a href="https://tutsplus.com/subscribe-with-elements?ec_promo=teaser_post_ebooks" target="_self">subscription to Envato Elements</a>. For a single low monthly fee, you get access not only to these eBooks, but also to our growing library of over 1,000 video courses on Envato Tuts+.</p><p>Plus you now get unlimited downloads from the huge Envato Elements library of 500,000+ creative assets. Create with unique fonts, photos, graphics and templates, and deliver better projects faster.</p>2018-04-18T11:37:58.000Z2018-04-18T11:37:58.000ZAndrew Blackmantag:code.tutsplus.com,2005:PostPresenter/cms-30286Charting Using Plotly in Python<p>Data visualization is a way to understand large chunks of data. Certain trends and patterns might go unnoticed in text format, so data visualization makes it easier to understand what the data is trying to say by visualizing it using different charts.</p><p>From the official documentation:</p><blockquote>
<a href="https://plot.ly/d3-js-for-python-and-pandas-charts/" rel="nofollow">plotly.py</a> is an interactive, browser-based graphing library for Python. Built on top of <a href="https://github.com/plotly/plotly.js">plotly.js</a>, plotly.py is a high-level, declarative charting library. plotly.js ships with over 30 chart types, including scientific charts, 3D graphs, statistical charts, SVG maps, financial charts, and more.</blockquote><p>In this tutorial, you'll be learning about the Plotly data visualization tool. You'll learn how to visualize data in Python using Plotly.</p><h2>Getting Started</h2><p>You'll be using a Python framework called Flask to create a Python web application. Once you have the application started, you'll see how to use the Plotly library to visualize data.</p><h3>Setting Up the Flask Web App </h3><p>Flask is a micro-framework for creating web applications using Python. It is quite easy to set up Flask. Install Flask using PIP.</p><pre class="brush: bash noskimlinks noskimwords">pip install flask</pre><p>Create a directory called <code class="inline">PythonPlot</code>. Navigate to the directory and create a file called <strong>app.py</strong>.</p><pre class="brush: bash noskimlinks noskimwords">mkdir PythonPlot
cd PythonPlot</pre><p>Add the following code to the <strong>app.py</strong> file.</p><pre class="brush: python noskimlinks noskimwords">from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
return "Welcome to TutsPlus!"</pre><p>Start the web application server using the following code:</p><pre class="brush: python noskimlinks noskimwords">FLASK_APP=app.py flask run</pre><p>Point your browser to <a href="http://localhost:5000/"><strong>http://localhost:5000/</strong></a> and you will have the web application running with the welcome message.</p><p>Now let's try to render an HTML page from your Flask web application. </p><p>Create a folder called <strong>templates</strong> and, inside the <strong>templates</strong> folder, create a file called <strong>index.html</strong>. You'll be rendering the graphs created using <code class="inline">plotly</code> in the <strong>index.html</strong> file.</p><p>Add the following HTML code to <strong>templates/index.html</strong>.</p><pre class="brush: html noskimlinks noskimwords">&lt;!doctype html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;style type="text/css"&gt;
.header{
text-align: center;
background-color: rgb(136, 185, 229);
height: 70px;
line-height: 70px;
}
.chart{
margin-top: 30px;
text-align: center;
}
&lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;div class="header"&gt;
&lt;h2&gt;
Plotly Chart Demo
&lt;/h2&gt;
&lt;/div&gt;
&lt;div id="chart" class="chart"&gt;
Chart will be here
&lt;/div&gt;
&lt;/body&gt;
&lt;/html&gt;
</pre><p>Import <code class="inline">render_template</code> inside the <strong>app.py</strong> file.</p><pre class="brush: python noskimlinks noskimwords">from flask import Flask, render_template</pre><p>Add a new route called <code class="inline">showLineChart</code> inside the <strong>app.py</strong> file. Here is how it looks:</p><pre class="brush: python noskimlinks noskimwords">@app.route('/showLineChart')
def line():
return render_template('index.html')</pre><p>Save the above changes and restart the server. Point your browser to <a href="http://localhost:5000/showLineChart"><strong>http://localhost:5000/showLineChart</strong></a>, and you will have the page rendered in your browser.</p><figure class="post_image"><img alt="Plotly Chart Display " data-src="https://cms-assets.tutsplus.com/uploads/users/1469/posts/30286/image/in.jpg"></figure><h2>Creating a Line Chart Using Plotly</h2><p>Let's get started with creating a line chart using Plotly. Import the plotly-related libraries in the <strong>app.py</strong> file.</p><pre class="brush: python noskimlinks noskimwords">import plotly
import plotly.plotly as py
import plotly.graph_objs as go</pre><p>You'll be using NumPy to generate random data for displaying inside the line chart. Import <code class="inline">numpy</code> in the <strong>app.py</strong> file.</p><pre class="brush: python noskimlinks noskimwords">import numpy as np</pre><p>You'll be using the <code class="inline">numpy.linspace</code> method to create evenly spaced samples calculated over the interval.</p><pre class="brush: python noskimlinks noskimwords">count = 500
xScale = np.linspace(0, 100, count)</pre><p>The above code creates 500 evenly spaced samples between 0 and 100 for the x-axis scale.</p><p>You can use <code class="inline">numpy.random.randn</code> to create random samples for the y-axis scale.</p><pre class="brush: python noskimlinks noskimwords">yScale = np.random.randn(count)</pre><p>Create a trace using the <code class="inline">plotly.graph_objs.scatter</code> method.</p><pre class="brush: python noskimlinks noskimwords">trace = go.Scatter(
x = xScale,
y = yScale
)</pre><p>You need to convert the trace into JSON format. For that, you'll make use of the plotly JSON encoder <code class="inline">plotly.utils.PlotlyJSONEncoder</code>.</p><pre class="brush: python noskimlinks noskimwords">data = [trace]
graphJSON = json.dumps(data, cls=plotly.utils.PlotlyJSONEncoder)</pre><p>Once you have the JSON data, you'll pass it to the template file to be rendered.</p><pre class="brush: python noskimlinks noskimwords">return render_template('index.html',
graphJSON=graphJSON)</pre><p>Here is how the <strong>app.py</strong> file looks:</p><pre class="brush: python noskimlinks noskimwords">from flask import Flask, render_template
import json
import plotly
import plotly.plotly as py
import plotly.graph_objs as go
import numpy as np
app = Flask(__name__)
@app.route('/showLineChart')
def line():
count = 500
xScale = np.linspace(0, 100, count)
yScale = np.random.randn(count)
# Create a trace
trace = go.Scatter(
x = xScale,
y = yScale
)
data = [trace]
graphJSON = json.dumps(data, cls=plotly.utils.PlotlyJSONEncoder)
return render_template('index1.html',
graphJSON=graphJSON)
</pre><p>You need to handle the JSON data on the client side to render the chart data. In the <strong>templates/index.html</strong> file, add references to the following scripts:</p><pre class="brush: html noskimlinks noskimwords">&lt;script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"&gt;&lt;/script&gt;
&lt;script src="https://code.jquery.com/jquery-2.1.4.min.js"&gt;&lt;/script&gt;
&lt;script src="https://d14fo0winaifog.cloudfront.net/plotly-basic.js"&gt;&lt;/script&gt;</pre><p>As seen in the above code, you have referenced the <code class="inline">plotly</code> script, as well as <code class="inline">jQuery</code> and <code class="inline">D3.js</code>, which are also required for <code class="inline">plotly</code> to work.</p><p>Add the following script to parse the passed-in JSON and render the chart.</p><pre class="brush: javascript noskimlinks noskimwords">var graphs = {{graphJSON | safe}};
Plotly.plot('chart',graphs,{}); </pre><p>The <code class="inline">safe</code> filter explicitly marks the string as safe, hence disabling auto-escaping. Once the JSON is parsed into the <code class="inline">graph</code> variable, you have passed it to the <code class="inline">plotly</code> <code class="inline">plot</code> method along with the ID of the div in which to render the line chart.</p><p>Here is how the <code class="inline">index.html</code> file looks:</p><pre class="brush: html noskimlinks noskimwords">&lt;!doctype html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;style type="text/css"&gt;
.header{
text-align: center;
background-color: rgb(136, 185, 229);
height: 70px;
line-height: 70px;
}
.chart{
margin-top: 30px;
text-align: center;
}
&lt;/style&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;div class="header"&gt;
&lt;h2&gt;
Plotly Chart Demo
&lt;/h2&gt;
&lt;/div&gt;
&lt;div id="chart" class="chart"&gt;
&lt;/div&gt;
&lt;/body&gt;
&lt;!-- D3.js --&gt;
&lt;script src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.6/d3.min.js"&gt;&lt;/script&gt;
&lt;script src="https://code.jquery.com/jquery-2.1.4.min.js"&gt;&lt;/script&gt;
&lt;!-- Plotly.js --&gt;
&lt;script src="https://d14fo0winaifog.cloudfront.net/plotly-basic.js"&gt;&lt;/script&gt;
&lt;script type="text/javascript"&gt;
var graphs = {{graphJSON | safe}};
Plotly.plot('chart',graphs,{});
&lt;/script&gt;
&lt;/html&gt;
</pre><p>Save the above changes and restart the server. Point your browser to <a href="http://localhost:5000/showLineChart"><strong>http://localhost:5000/showLineChart</strong></a>, and you will have the line chart rendered.</p><figure class="post_image"><img alt="Line Chart Using Plotly" data-src="https://cms-assets.tutsplus.com/uploads/users/1469/posts/30286/image/lin.jpg"></figure><h2>Creating a Multi-Line Chart Using Plotly</h2><p>With some modifications to the above line chart, you can convert it into a multi-line chart. To create a multi-line chart, you need to add extra y-axis scales.</p><p>Let's start by creating a new route for displaying the multi-line chart.</p><pre class="brush: python noskimlinks noskimwords">@app.route('/showMultiChart')
def multiLine():</pre><p>Create an x-axis scale, as you did when creating the line chart, and add three y-axis scales.</p><pre class="brush: python noskimlinks noskimwords">count = 500
xScale = np.linspace(0, 100, count)
y0_scale = np.random.randn(count)
y1_scale = np.random.randn(count)
y2_scale = np.random.randn(count)</pre><p>Create traces using the above <code class="inline">xScale</code> and each of the y scales.</p><pre class="brush: python noskimlinks noskimwords">trace0 = go.Scatter(
x = xScale,
y = y0_scale
)
trace1 = go.Scatter(
x = xScale,
y = y1_scale
)
trace2 = go.Scatter(
x = xScale,
y = y2_scale
)</pre><p>Convert the data into JSON using the plotly <code class="inline">json</code> encoder, as you did when creating a single line chart.</p><pre class="brush: python noskimlinks noskimwords">data = [trace0, trace1, trace2]
graphJSON = json.dumps(data, cls=plotly.utils.PlotlyJSONEncoder)
return render_template('index.html',
graphJSON=graphJSON)</pre><p>Here is what the <code class="inline">/showMultiChart</code> routing looks like:</p><pre class="brush: python noskimlinks noskimwords">@app.route('/showMultiChart')
def multiLine():
count = 500
xScale = np.linspace(0, 100, count)
y0_scale = np.random.randn(count)
y1_scale = np.random.randn(count)
y2_scale = np.random.randn(count)
# Create traces
trace0 = go.Scatter(
x = xScale,
y = y0_scale
)
trace1 = go.Scatter(
x = xScale,
y = y1_scale
)
trace2 = go.Scatter(
x = xScale,
y = y2_scale
)
data = [trace0, trace1, trace2]
graphJSON = json.dumps(data, cls=plotly.utils.PlotlyJSONEncoder)
return render_template('index1.html',
graphJSON=graphJSON)</pre><p>Save the above changes and restart the server. Point your browser to <a href="http://localhost:5000/showMultiChart"><strong>http://localhost:5000/showMultiChart</strong></a>, and you will have the multi-line chart rendered.</p><figure class="post_image"><img alt="Multi Line Chart Using Plotly" data-src="https://cms-assets.tutsplus.com/uploads/users/1469/posts/30286/image/ml.jpg"></figure><h2>Wrapping It Up</h2><p>In this tutorial, you learned how to create line and multi-line charts in Python using the Plotly library. You created a Python Flask web app and saw how to create a line chart using sample data generated with the NumPy library.</p><p>You can do a lot more using Plotly. For detailed information, I would recommend reading <a href="https://plot.ly/python/" rel="external" target="_blank">the official documentation</a>.</p><p>Source code from this tutorial is available in <a href="https://github.com/tutsplus/charting-using-plotly-in-python" rel="external" target="_blank">the tutorial GitHub repo</a>.</p><p>How was your experience learning to create charts using Plotly? Do let us know your thoughts and suggestions in the comments below.</p>2018-04-12T16:13:36.506Z2018-04-12T16:13:36.506ZRoy Agasthyantag:code.tutsplus.com,2005:PostPresenter/cms-30486Modern Web Scraping With BeautifulSoup and Selenium<h2>Overview</h2><p>HTML is almost intuitive. CSS is a great advancement that cleanly separates the structure of a page from its look and feel. JavaScript adds some pizazz. That's the theory. The real world is a little different. <br></p><p>In this tutorial, you'll learn how the content you see in the browser actually gets rendered and how to go about scraping it when necessary. In particular, you'll learn how to count Disqus comments. Our tools will be Python and awesome packages like requests, BeautifulSoup, and Selenium.<br></p><h2>When Should You Use Web Scraping?</h2><p>Web scraping is the practice of automatically fetching the content of web pages designed for interaction with human users, parsing them, and extracting some information (possibly navigating links to other pages). It is sometimes necessary if there is no other way to extract the necessary information. Ideally, the application provides a dedicated API for accessing its data programmatically. There are several reasons web scraping should be your last resort:</p><ul>
<li>It is fragile (the web pages you're scraping might change frequently).</li>
<li>It might be forbidden (some web apps have policies against scraping).</li>
<li>It might be slow and expansive (if you need to fetch and wade through a lot of noise).</li>
</ul><h2>Understanding Real-World Web Pages</h2><p>Let's understand what we are up against, by looking at the output of some common web application code. In the article <a href="https://code.tutsplus.com/tutorials/introduction-to-vagrant--cms-25917" target="_self">Introduction to Vagrant</a>, there are some Disqus comments at the bottom of the page:</p><figure class="post_image"><img alt="Understanding Real-World Web Pages" data-src="https://cms-assets.tutsplus.com/uploads/users/1199/posts/30486/image/disqus_comments.png"></figure><p>In order to scrape these comments, we need to find them on the page first.<br></p><h3>View Page Source</h3><p>Every browser since the dawn of time (the 1990s) has supported the ability to view the HTML of the current page. Here is a snippet from the view source of <a href="https://code.tutsplus.com/tutorials/introduction-to-vagrant--cms-25917" target="_self">Introduction to Vagrant</a> that starts with a huge chunk of minified and uglified JavaScript unrelated to the article itself. Here is a <em>small</em> portion of it:</p><figure class="post_image"><img alt="Page Source" data-src="https://cms-assets.tutsplus.com/uploads/users/1199/posts/30486/image/view_page_source.png"></figure><p>Here is some actual HTML from the page:</p><figure class="post_image"><img alt="HTML From the Page" data-src="https://cms-assets.tutsplus.com/uploads/users/1199/posts/30486/image/some_html.png"></figure><p>This looks pretty messy, but what is surprising is that you will not find the Disqus comments in the source of the page.</p><h3>The Mighty Inline Frame</h3><p>It turns out that the page is a mashup, and the Disqus comments are embedded as an iframe (inline frame) element. You can find it out by right-clicking on the comments area, and you'll see that there is frame information and source there:</p><figure class="post_image"><img alt="The Mighty Inline Frame" data-src="https://cms-assets.tutsplus.com/uploads/users/1199/posts/30486/image/this_frame.png"></figure><p>That makes sense. Embedding third-party content as an iframe is one of the primary reasons to use iframes. Let's find the <code class="inline">&lt;iframe&gt;</code> tag then in the main page source. Foiled again! There is no <code class="inline">&lt;iframe&gt;</code> tag in the main page source. </p><h3>JavaScript-Generated Markup</h3><p>The reason for this omission is that <code class="inline">view page source</code> shows you the content that was fetched from the server. But the final DOM (document object model) that gets rendered by the browser may be very different. JavaScript kicks in and can manipulate the DOM at will. The iframe can't be found, because it wasn't there when the page was retrieved from the server. </p><h2>Static Scraping vs. Dynamic Scraping</h2><p>Static scraping ignores JavaScript. It fetches web pages from the server without the help of a browser. You get exactly what you see in "view page source", and then you slice and dice it. If the content you're looking for is available, you need to go no further. However, if the content is something like the Disqus comments iframe, you need dynamic scraping. </p><p>Dynamic scraping uses an actual browser (or a headless browser) and lets JavaScript do its thing. Then, it queries the DOM to extract the content it's looking for. Sometimes you need to automate the browser by simulating a user to get the content you need.</p><h2>Static Scraping With Requests and BeautifulSoup</h2><p>Let's see how static scraping works using two awesome Python packages: <a href="http://docs.python-requests.org/en/master/" target="_self">requests</a> for fetching web pages and <a href="https://www.crummy.com/software/BeautifulSoup/" target="_self">BeautifulSoup</a> for parsing HTML pages.</p><h3>Installing Requests and BeautifulSoup</h3><p>Install <a href="https://docs.pipenv.org/" target="_self">pipenv</a> first, and then: <code class="inline">pipenv install requests beautifulsoup4</code> </p><p>This will create a virtual environment for you too. If you're using the code from gitlab, you can just <code class="inline">pipenv install</code>.<br></p><h3>Fetching Pages</h3><p>Fetching a page with requests is a one liner: <code class="inline">r = requests.get(url)</code></p><p>The response object has a lot of attributes. The most important ones are <code class="inline">ok</code> and <code class="inline">content</code>. If the request fails then <code class="inline">r.ok</code> will be False and <code class="inline">r.content</code> will contain the error. The content is a stream of bytes. It is usually better to decode it to utf-8 when dealing with text:</p><pre class="brush: python noskimlinks noskimwords">&gt;&gt;&gt; r = requests.get('http://www.c2.com/no-such-page')
&gt;&gt;&gt; r.ok
False
&gt;&gt;&gt; print(r.content.decode('utf-8'))
&lt;!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"&gt;
&lt;html&gt;&lt;head&gt;
&lt;title&gt;404 Not Found&lt;/title&gt;
&lt;/head&gt;&lt;body&gt;
&lt;h1&gt;Not Found&lt;/h1&gt;
&lt;p&gt;The requested URL /ggg was not found on this server.&lt;/p&gt;
&lt;hr&gt;
&lt;address&gt;
Apache/2.0.52 (CentOS) Server at www.c2.com Port 80
&lt;/address&gt;
&lt;/body&gt;&lt;/html&gt;
</pre><p>If everything is OK then <code class="inline">r.content</code> will contain the requested web page (same as view page source).</p><h3>Finding Elements With BeautifulSoup</h3><p>The <code class="inline">get_page()</code> function below fetches a web page by URL, decodes it to UTF-8, and parses it into a BeautifulSoup object using the HTML parser.</p><pre class="brush: python noskimlinks noskimwords">def get_page(url):
r = requests.get(url)
content = r.content.decode('utf-8')
return BeautifulSoup(content, 'html.parser')
</pre><p>Once we have a BeautifulSoup object, we can start extracting information from the page. BeautifulSoup provides many find functions to locate elements inside the page and drill down deep nested elements. </p><p>Tuts+ author pages contain multiple tutorials. Here is my <a href="https://tutsplus.com/authors/gigi-sayfan" target="_self">author page</a>. On each page, there are up to 12 tutorials. If you have more than 12 tutorials then you can navigate to the next page. The HTML for each article is enclosed in an <code class="inline">&lt;article&gt;</code> tag. The following function finds all the article elements on the page, drills down to their links, and extracts the href attribute to get the URL of the tutorial:</p><pre class="brush: python noskimlinks noskimwords">def get_page_articles(page):
elements = page.findAll('article')
articles = [e.a.attrs['href'] for e in elements]
return articles
</pre><p>The following code gets all the articles from my page and prints them (without the common prefix):</p><pre class="brush: python noskimlinks noskimwords">page = get_page('https://tutsplus.com/authors/gigi-sayfan')
articles = get_page_articles(page)
prefix = 'https://code.tutsplus.com/tutorials'
for a in articles:
print(a[len(prefix):])
Output:
building-games-with-python-3-and-pygame-part-5--cms-30085
building-games-with-python-3-and-pygame-part-4--cms-30084
building-games-with-python-3-and-pygame-part-3--cms-30083
building-games-with-python-3-and-pygame-part-2--cms-30082
building-games-with-python-3-and-pygame-part-1--cms-30081
mastering-the-react-lifecycle-methods--cms-29849
testing-data-intensive-code-with-go-part-5--cms-29852
testing-data-intensive-code-with-go-part-4--cms-29851
testing-data-intensive-code-with-go-part-3--cms-29850
testing-data-intensive-code-with-go-part-2--cms-29848
testing-data-intensive-code-with-go-part-1--cms-29847
make-your-go-programs-lightning-fast-with-profiling--cms-29809</pre><h2>Dynamic Scraping With Selenium</h2><p>Static scraping was good enough to get the list of articles, but as we saw earlier, the Disqus comments are embedded as an iframe element by JavaScript. In order to harvest the comments, we will need to automate the browser and interact with the DOM interactively. One of the best tools for the job is <a href="http://selenium-python.readthedocs.io/" target="_self">Selenium</a>. <br></p><p>Selenium is primarily geared towards automated testing of web applications, but it is great as a general-purpose browser automation tool.</p><h3>Installing Selenium</h3><p>Type this command to install Selenium: <code class="inline">pipenv install selenium</code><br></p><h3>Choose Your Web Driver</h3><p>Selenium needs a web driver (the browser it automates). For web scraping, it usually doesn't matter which driver you choose. I prefer the Chrome driver. Follow the instructions in <a href="http://selenium-python.readthedocs.io/installation.html#drivers" target="_self">this Selenium guide</a>.</p><h3>Chrome vs. PhantomJS</h3><p>In some cases you may prefer to use a headless browser, which means no UI is displayed. Theoretically, PhantomJS is just another web driver. But, in practice, people reported incompatibility issues where Selenium works properly with Chrome or Firefox and sometimes fails with PhantomJS. I prefer to remove this variable from the equation and use an actual browser web driver.<br></p><h2>Counting Disqus Comments</h2><p>Let's do some dynamic scraping and use Selenium to count Disqus comments on Tuts+ tutorials. Here are the necessary imports.</p><pre class="brush: python noskimlinks noskimwords">from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.expected_conditions import (
presence_of_element_located)
from selenium.webdriver.support.wait import WebDriverWait</pre><p>The <code class="inline">get_comment_count()</code> function accepts a Selenium driver and URL. It uses the <code class="inline">get()</code> method of the driver to fetch the URL. This is similar to <code class="inline">requests.get()</code>, but the difference is that the driver object manages a live representation of the DOM.</p><p>Then, it gets the title of the tutorial and locates the Disqus iframe using its parent id <code class="inline">disqus_thread</code> and then the iframe itself:</p><pre class="brush: python noskimlinks noskimwords">def get_comment_count(driver, url):
driver.get(url)
class_name = 'content-banner__title'
name = driver.find_element_by_class_name(class_name).text
e = driver.find_element_by_id('disqus_thread')
disqus_iframe = e.find_element_by_tag_name('iframe')
iframe_url = disqus_iframe.get_attribute('src')
</pre><p>The next step is to fetch the contents of the iframe itself. Note that we wait for the <code class="inline">comment-count</code> element to be present because the comments are loaded dynamically and not necessarily available yet.</p><pre class="brush: python noskimlinks noskimwords"> driver.get(iframe_url)
wait = WebDriverWait(driver, 5)
commentCountPresent = presence_of_element_located(
(By.CLASS_NAME, 'comment-count'))
wait.until(commentCountPresent)
comment_count_span = driver.find_element_by_class_name(
'comment-count')
comment_count = int(comment_count_span.text.split()[0])
</pre><p>The last part is to return the last comment if it wasn't made by me. The idea is to detect comments I haven't responded to yet.</p><pre class="brush: python noskimlinks noskimwords"> last_comment = {}
if comment_count &gt; 0:
e = driver.find_elements_by_class_name('author')[-1]
last_author = e.find_element_by_tag_name('a')
last_author = e.get_attribute('data-username')
if last_author != 'the_gigi':
e = driver.find_elements_by_class_name('post-meta')
meta = e[-1].find_element_by_tag_name('a')
last_comment = dict(
author=last_author,
title=meta.get_attribute('title'),
when=meta.text)
return name, comment_count, last_comment</pre><h2>Conclusion</h2><p>Web scraping is a useful practice when the information you need is accessible through a web application that doesn't provide an appropriate API. It takes some non-trivial work to extract data from modern web applications, but mature and well-designed tools like requests, BeautifulSoup, and Selenium make it worthwhile.</p><p>Additionally, don’t hesitate to see what we have available for sale and for study <a href="https://codecanyon.net/search/python" target="_self">in the Envato Market</a>, and don't hesitate to ask any questions and provide your valuable feedback using the feed below.</p>2018-03-23T12:00:00.000Z2018-03-23T12:00:00.000ZGigi Sayfantag:code.tutsplus.com,2005:PostPresenter/cms-30460JWT Authentication in Django<p>This tutorial will give an introduction to JSON Web Tokens (JWT) and how to implement JWT authentication in Django.<br></p><h2>What Is JWT?</h2>
<p>JWT is an encoded JSON string that is passed in headers to authenticate requests. It is usually obtained by hashing JSON data with a secret key. This means that the server doesn't need to query the database every time to retrieve the user associated with a given token.</p>
<h2>How JSON Web Tokens Work</h2><p>When a user successfully logs in using their credentials, a JSON Web Token is obtained and saved in local storage. Whenever the user wants to access a protected URL, the token is sent in the header of the request. The server then checks for a valid JWT in the Authorization header, and if found, the user will be allowed access.</p>
<p>A typical content header will look like this:</p>
<p><code class="inline">Authorization:
Bearer eyJhbGciOiJIUzI1NiIsI <br></code></p><p>Below is a diagram showing this process:</p><figure class="post_image"><img alt="How JSON Web Tokens Work" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/jwt-diagram.png"></figure><h2>The Concept of Authentication and Authorization</h2>
<p>Authentication is the process of identifying a logged-in user, while authorization is the process of identifying if a certain user has the right to access a web resource.</p><h2>API Example</h2><p>In this tutorial, we are going to build a simple user authentication system in Django using JWT as the authentication mechanism.</p>
<h3>Requirements</h3><ul>
<li>Django</li>
<li>Python</li>
</ul>
<p>Let's get started.</p><p>Create a directory where you will keep your project and also a virtual environment to install the project dependencies.</p><pre class="brush: bash noskimlinks noskimwords">mkdir myprojects
cd myprojects
virtual venv
</pre><p>Activate the virtual environment:<br></p><pre class="brush: bash noskimlinks noskimwords">source venv/bin/activate
</pre><p>Create a Django project.</p><pre class="brush: bash noskimlinks noskimwords">django-admin startproject django_auth
</pre><p>Install DRF and django-rest-framework-jwt using pip. <br></p><pre class="brush: bash noskimlinks noskimwords">pip install djangorestframework
pip install djangorestframework-jwt
pip install django</pre><p>Let's go ahead and add DRF to the list of installed apps in the <code class="inline">settings.py</code> file.</p><h3>Configure the JWT Settings</h3>
<p>In order to use JWT, we need to configure django-rest-framework permissions to accept JSON Web Tokens.</p>
<p>In the <code class="inline">settings.py</code> file, add the following configurations:</p><pre class="brush: python noskimlinks noskimwords">REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
),
}</pre> <p>Create a new app called users which will handle user authentication and management.</p><pre class="brush: python noskimlinks noskimwords">cd django-auth
django-admin.py startapp users </pre> <p>Add the users application to the list of installed apps in the <code class="inline">settings.py</code> file.</p><h3>Setting Up the Database</h3>
<p>We are going to use the PostgreSQL database because it's more stable and robust.</p><p>Create the <code class="inline">auth</code> database and assign a user.</p><p>Switch over to the Postgres account on your machine by typing:</p>
<pre class="brush: bash noskimlinks noskimwords">sudo su postgres</pre>
<p>Access the Postgres prompt and create the database:</p><pre class="brush: bash noskimlinks noskimwords">psql
postgres=# CREATE DATABASE auth;</pre><p>Create a role:</p><pre class="brush: bash noskimlinks noskimwords">postgres=# CREATE ROLE django_auth WITH LOGIN PASSWORD 'asdfgh'; </pre><p>Grant database access to the user:</p>
<pre class="brush: bash noskimlinks noskimwords">postgres=# GRANT ALL PRIVILEGES ON DATABASE auth TO django_auth;</pre>
<p>Install the psycopg2 package, which will allow us to use the database we configured:</p><pre class="brush: bash noskimlinks noskimwords">pip install psycopg2</pre>
<p>Edit the currently configured SQLite database and use the Postgres database.</p><pre class="brush: bash noskimlinks noskimwords">DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'auth',
'USER': 'django_auth',
'PASSWORD': 'asdfgh',
'HOST': 'localhost',
'PORT': '',
}
}</pre> <h3>Creating Models</h3>
<p>Django comes with a built-in authentication system which is very elaborate, but sometimes we need to make adjustments, and thus we need to create a custom user authentication system. Our user model will be inheriting from the <code class="inline">AbstractBaseUser</code> class provided by <code class="inline">django.contrib.auth.models</code>.</p>
<p>In users/models.py, we start by creating the User model to store the user details.</p><pre class="brush: python noskimlinks noskimwords"># users/models.py
from __future__ import unicode_literals
from django.db import models
from django.utils import timezone
from django.contrib.auth.models import (
AbstractBaseUser, PermissionsMixin
)
class User(AbstractBaseUser, PermissionsMixin):
"""
An abstract base class implementing a fully featured User model with
admin-compliant permissions.
"""
email = models.EmailField(max_length=40, unique=True)
first_name = models.CharField(max_length=30, blank=True)
last_name = models.CharField(max_length=30, blank=True)
is_active = models.BooleanField(default=True)
is_staff = models.BooleanField(default=False)
date_joined = models.DateTimeField(default=timezone.now)
objects = UserManager()
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = ['first_name', 'last_name']
def save(self, *args, **kwargs):
super(User, self).save(*args, **kwargs)
return self
</pre> <p><code class="inline">REQUIRED_FIELDS</code> contains all required fields on your user model, except the username field and password, as these fields will always be prompted for. <br></p><p><code class="inline">UserManager</code> is the class that defines the <code class="inline">create_user</code> and <code class="inline">createsuperuser</code> methods. This class should come before the <code class="inline">AbstractBaseUser</code> class we defined above. Let's go ahead and define it.</p><pre class="brush: python noskimlinks noskimwords">from django.contrib.auth.models import (
AbstractBaseUser, PermissionsMixin, BaseUserManager
)
class UserManager(BaseUserManager):
def _create_user(self, email, password, **extra_fields):
"""
Creates and saves a User with the given email,and password.
"""
if not email:
raise ValueError('The given email must be set')
try:
with transaction.atomic():
user = self.model(email=email, **extra_fields)
user.set_password(password)
user.save(using=self._db)
return user
except:
raise
def create_user(self, email, password=None, **extra_fields):
extra_fields.setdefault('is_staff', False)
extra_fields.setdefault('is_superuser', False)
return self._create_user(email, password, **extra_fields)
def create_superuser(self, email, password, **extra_fields):
extra_fields.setdefault('is_staff', True)
extra_fields.setdefault('is_superuser', True)
return self._create_user(email, password=password, **extra_fields)</pre> <h4>Migrations</h4>
<p>Migrations provide a way of updating your database schema every time your models change, without losing data.</p>
<p>Create an initial migration for our users model, and sync the database for the first time. <br></p><pre class="brush: bash noskimlinks noskimwords">python manage.py make migrations users
python manage.py migrate</pre><h4>Creating a Superuser</h4><p>Create a superuser by running the following command:</p>
<pre class="brush: bash noskimlinks noskimwords">python manage.py createsuperuser</pre><h3>Creating New Users</h3>
<p>Let's create an endpoint to enable registration of new users. We will start by serializing the User model fields. Serializers provide a way of changing data to a form that is easier to understand, like JSON or XML. Deserialization does the opposite, which is converting data to a form that can be saved to the database.</p>
<p>Create users/serializers.py and add the following code.</p><pre class="brush: python noskimlinks noskimwords"># users/serializers.py
from rest_framework import serializers
from.models import User
class UserSerializer(serializers.ModelSerializer):
date_joined = serializers.ReadOnlyField()
class Meta(object):
model = User
fields = ('id', 'email', 'first_name', 'last_name',
'date_joined', 'password')
extra_kwargs = {'password': {'write_only': True}}</pre> <h4>CreateUserAPIView</h4>
<p>Next, we want to create a view so the client will have a URL for creating new users.</p>
<p>In users.views.py, add the following:</p><pre class="brush: python noskimlinks noskimwords"># users/views.py
class CreateUserAPIView(APIView):
# Allow any user (authenticated or not) to access this url
permission_classes = (AllowAny,)
def post(self, request):
user = request.data
serializer = UserSerializer(data=user)
serializer.is_valid(raise_exception=True)
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)</pre> <p>We set <code class="inline">permission_classes</code> to <code class="inline">(AllowAny,)</code> to allow any user (authenticated or not) to access this URL.</p>
<h4>Configuring URLs</h4>
<p>Create a file <code class="inline">users/urls.py</code> and add the URL to match the view we created. Also add the following code.</p><pre class="brush: python noskimlinks noskimwords"># users/urls.py
from django.conf.urls import url, patterns
from .views import CreateUserAPIView
urlpatterns = [
url(r'^create/$', CreateUserAPIView.as_view()),
]</pre> <p>We also need to import URLs from the users application to the main <code class="inline">django_auth/urls.py</code> file. So go ahead and do that. We are using the <code class="inline">include</code> function here, so don't forget to import it.</p><pre class="brush: python noskimlinks noskimwords"># django_auth/urls.py
from django.conf.urls import url, include
from django.contrib import admin
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^user/', include('users.urls', namespace='users')),
]
</pre> <p>Now that we are done creating the endpoint, let's do a test and see if we are on track. We will use Postman to do the tests. If you are not familiar with Postman, it's a tool which presents a friendly GUI for constructing requests and reading responses.</p><figure class="post_image"><img alt="Configuring URLs" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/create.png"></figure><p>As you can see above, the endpoint is working as expected.</p>
<h3>Authenticating Users</h3>
<p>We will make use of the Django-REST Framework JWT Python module we installed at the beginning of this tutorial. It adds JWT authentication support for Django Rest Framework apps.</p>
<p>But first, let's define some configuration parameters for our tokens and how they are generated in the settings.py file.</p><pre class="brush: python noskimlinks noskimwords"># settings.py
import datetime
JWT_AUTH = {
'JWT_VERIFY': True,
'JWT_VERIFY_EXPIRATION': True,
'JWT_EXPIRATION_DELTA': datetime.timedelta(seconds=3000),
'JWT_AUTH_HEADER_PREFIX': 'Bearer',
}</pre> <ul>
<li>
<code class="inline">JWT_VERIFY</code>: It will raise a jwt.DecodeError if the secret is wrong.</li>
<li>
<code class="inline">JWT_VERIFY_EXPIRATION</code>: Sets the expiration to True, meaning Tokens will expire after a period of time. The default time is five minutes.</li>
<li>
<code class="inline">JWT_AUTH_HEADER_PREFIX</code>: The Authorization header value prefix that is required to be sent together with the token. We have set it as <code class="inline">Bearer</code>, and the default is <code class="inline">JWT</code>.</li>
</ul><p>In <code class="inline">users/views.py</code>, add the following code.</p><pre class="brush: python noskimlinks noskimwords">@api_view(['POST'])
@permission_classes([AllowAny, ])
def authenticate_user(request):
try:
email = request.data['email']
password = request.data['password']
user = User.objects.get(email=email, password=password)
if user:
try:
payload = jwt_payload_handler(user)
token = jwt.encode(payload, settings.SECRET_KEY)
user_details = {}
user_details['name'] = "%s %s" % (
user.first_name, user.last_name)
user_details['token'] = token
user_logged_in.send(sender=user.__class__,
request=request, user=user)
return Response(user_details, status=status.HTTP_200_OK)
except Exception as e:
raise e
else:
res = {
'error': 'can not authenticate with the given credentials or the account has been deactivated'}
return Response(res, status=status.HTTP_403_FORBIDDEN)
except KeyError:
res = {'error': 'please provide a email and a password'}
return Response(res)</pre> <p>In the code above, the login view takes username and password as input, and it then creates a token with the user information corresponding to the passed credentials as payload and returns it to the browser. Other user details such as name are also returned to the browser together with the token. This token will be used to authenticate in future requests.</p>
<p>The permission classes are set to <code class="inline">allowAny</code> since anyone can access this endpoint.</p>
<p>We also store the last login time of the user with this code.</p><pre class="brush: python noskimlinks noskimwords">user_logged_in.send(sender=user.__class__,
request=request, user=user)</pre> <p>Every time the user wants to make an API request, they have to send the token in Auth Headers in order to authenticate the request.</p>
<p>Let's test this endpoint with Postman. Open Postman and use the request to authenticate with one of the users you created previously. If the login attempt is successful, the response will look like this:</p><figure class="post_image"><img alt="A sample response" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/token(1).png"></figure><h4>Retrieving and Updating Users</h4><p>So far, users can register and authenticate themselves. However, they also need a way to retrieve and update their information. Let's implement this.</p>
<p>In <code class="inline">users.views.py</code>, add the following code.</p><pre class="brush: python noskimlinks noskimwords">class UserRetrieveUpdateAPIView(RetrieveUpdateAPIView):
# Allow only authenticated users to access this url
permission_classes = (IsAuthenticated,)
serializer_class = UserSerializer
def get(self, request, *args, **kwargs):
# serializer to handle turning our `User` object into something that
# can be JSONified and sent to the client.
serializer = self.serializer_class(request.user)
return Response(serializer.data, status=status.HTTP_200_OK)
def put(self, request, *args, **kwargs):
serializer_data = request.data.get('user', {})
serializer = UserSerializer(
request.user, data=serializer_data, partial=True
)
serializer.is_valid(raise_exception=True)
serializer.save()
return Response(serializer.data, status=status.HTTP_200_OK)</pre>
<p>We first define the permission classes and set to <code class="inline">IsAuthenticated</code> since this is a protected URL and only authenticated users can access it. </p><p>We then define a <code class="inline">get</code> method to retrieve user details. After retrieving user details, an authenticated user will then update their details as desired.<br></p>
<p>Update your URLs to define the endpoint as follows.</p><pre class="brush: python noskimlinks noskimwords">users/urls.py
from .views import CreateUserAPIView, UserRetrieveUpdateAPIView
urlpatterns = [
url(r'^update/$', UserRetrieveUpdateAPIView.as_view()),
]</pre><p>In order for the request to be successful, the headers should contain the JWT token as shown below.</p><figure class="post_image"><img alt="Headers with the JWT Token" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/update.png"></figure>
<p>If you attempt to request a resource without the authentication header, you will get the following error.</p><figure class="post_image"><img alt="An example error" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/e1c6iexptiizbh3m0a2p_error.png"></figure><p>If a user stays beyond the time specified in <code class="inline">JWT_EXPIRATION_DELTA</code> without making a request, the token will expire and they will have to request another token. This is also demonstrated below.</p><figure class="post_image"><img alt="JWT_EXPIRATION_DELTA Example" data-src="https://cms-assets.tutsplus.com/uploads/users/1885/posts/30460/image/expire.png"></figure><h2>Conclusion</h2>
<p>This tutorial has covered what is necessary to successfully build a solid back-end authentication system with JSON Web Tokens.</p>2018-03-14T12:00:00.000Z2018-03-14T12:00:00.000ZEsther Vaatitag:code.tutsplus.com,2005:PostPresenter/cms-30418Demystifying Python Recursion<p>Most complex tasks in Python can be broken down into simpler subtasks. Recursion helps to achieve this, hence making the code clean and neat. This tutorial will introduce recursion, the benefits of recursion, and how to use it in Python programming.<br></p><h2>What Is Recursion?</h2><p>Recursion is a method of solving a problem with the solutions to smaller instances of the same problem. This approach can be applied to many types of challenges in programming.</p><h2>The Benefits of Using Recursion</h2><p>Some of the benefits of using recursion are:</p><ul>
<li>Recursion adds simplicity when writing code, hence making it easier to debug.</li>
<li>Recursion reduces the amount of time taken by an algorithm to run as a function of the length of the input.</li>
<li>Recursion is also preferred when solving very complex problems, especially problems on tree-based structures, because it performs better.</li>
</ul><h2>Introduction to the Python Recursive Function</h2><p>Although recursion seems like a complicated procedure, it's not all that complicated. In layman's terms, assume you have two rectangles A and B. If you add them together, they form a rectangle C. This is in itself a recursive procedure. We have used smaller instances of a rectangle to define itself, and if we were to write a Python function, it would be as follows:</p><pre class="brush: python noskimlinks noskimwords">def rectangle(a,b):
return a+b
</pre><p><span class="wysiwyg-font-size-medium">Since a recursive function calls itself, there needs to be a rule or a breakpoint at which the process or loop would terminate. Such a condition is known as a base condition. A base condition is a requirement in every recursive program, otherwise the procedure would result in an infinite loop.<br></span></p><p><span class="wysiwyg-font-size-medium">The second requirement is the </span>recursive case when the function calls itself. <br></p><p><span class="wysiwyg-font-size-medium">Let's look at an example:<br></span></p><p><span class="wysiwyg-font-size-medium">In this example, you will write a factorial function that takes an integer (positive) as an input. The factorial of a number is obtained by multiplying the number by all positive integers below it. For example, <code class="inline">factorial(3) = 3 x 2 x 1</code>, <code class="inline">factorial(2) = 2 x 1</code>, and <code class="inline">factorial(0) = 1</code>.</span></p><p><span class="wysiwyg-font-size-medium">The first thing to do is to define our base case, which will be factorial(0) = 1.<br></span></p><p><span class="wysiwyg-font-size-medium">As you can see above, there is a relationship between each consecutive factorial scenario. You should notice that factorial(4) = 4 x factorial(3). Similarly, factorial(5) = 5 x factorial(4). <br></span></p><p><span class="wysiwyg-font-size-medium">The second part will be writing a function that calls itself.<br></span></p><p><span class="wysiwyg-font-size-medium">Now that we have simplified it, the resulting function will be:<br></span></p><pre class="brush: python noskimlinks noskimwords">def factorial(n):
if(n == 0):
#Define our base case?
return 1
else:
return n*factorial(n-1)
print(factorial(5))
#result
# 120</pre><p><span class="wysiwyg-font-size-medium">The solution if <code class="inline">n==0</code> is:<br></span></p><pre class="brush: python noskimlinks noskimwords">def factorial(n):
if(n == 0):
#Define our base case?
return 1
else:
return n*factorial(n-1)
print(factorial(0))
#result
# 0</pre><p><span class="wysiwyg-font-size-medium">Now that you know how to write recursive functions, let's look at several case studies that will solidify your understanding of recursion.</span></p><h3><span class="wysiwyg-font-size-medium">Case Study 1: Fibonacci</span></h3><p>In a Fibonacci sequence, each number is the sum of the two preceding numbers, such as: 1 + 1 = 2; 1 + 2 = 3; 2 + 3 = 5; 3 + 5 = 8. The Fibonacci sequence has been applied in many areas, and the most common is in predicting price action in the stock market by forex traders.</p><p>The Fibonacci sequence starts with 0 and 1. The first number in a Fibonacci sequence is 0, the second number is 1, and the third term in the sequence is 0 + 1 = 1. The fourth is 1 + 1 = 2 and so on. <br></p><p>In order to come up with a recursive function, you need to have two base cases, i.e. 0 and 1. You can then translate the adding pattern into the else case. <br></p><p>The resulting function will be:</p><pre class="brush: python noskimlinks noskimwords">def fibonacci(n):
if(n == 1):
#define Base case 1
return 0+1
elif(n == 2):
#define Base case 1
return 1+2
else:
return fibonacci(n) + fibonacci(n-1)
print(fibonacci(5))
#result
# </pre><h3>Case Study 2: Reversing a String</h3><p>In this example, you will write a function that takes a string as input and then returns the reverse of the string. <br></p><p>The first thing to do is to define our base case, which will check if the string is equal to 0 and, if so, will return the string itself.</p><p>The second step is to recursively call the reverse function to slice the part of the string excluding the first character and then concatenate the first character to the end of the sliced string.</p><p>The resulting function is as shown below:</p><pre class="brush: python noskimlinks noskimwords">def reverse(a):
if len(a) == 0:
return a
else:
return reverse(a[1:]) + a[0]
print(reverse("Python is a very easy language to learn"))
# result
#nrael ot egaugnal ysae yrev a si nohtyP
</pre><h3>Case study 3: Sum of Elements</h3><p>In this example, you will write a function that takes an array as input and then returns the sum of the elements in the list. <br></p><p>The first thing to do is to define our base case, which will check if the size of the list is zero, and return 0 if True.<br></p><p>The second step returns the element and a call to the function sum() minus one element of the list. <br></p><p>The solution is as shown below:<br></p><pre class="brush: python noskimlinks noskimwords">def sum_of_numbers(l):
if len(l) == 1:
return 0
else:
return l[0] + sum(l[1:])
a =[5,7,3,8,10]
print(sum(a))
# result
# 33</pre><p>The solution for an empty list is as shown below:</p><pre class="brush: python noskimlinks noskimwords">def sum_of_numbers(l):
if len(l) == 1:
return 0
else:
return l[0] + sum(l[1:])
b =[]
print(sum(b))
# result
# 0</pre><h2>Conclusion</h2><p>This tutorial has covered what is necessary to use recursion to solve complex programs in Python. It's also important to note that recursion has its own limitations:</p><ul class="features">
<li>Recursion takes a lot of stack space, hence making it a bit slow to maintain the program.</li>
<li>Recursion functions require more space and time to execute.</li>
</ul><p>Remember, don’t hesitate to see what we have available for sale and for study <a href="https://codecanyon.net/search/python" target="_self">in the Envato Market</a>, and please ask any questions and provide your valuable feedback using the feed below.</p>2018-03-07T12:00:00.000Z2018-03-07T12:00:00.000ZEsther Vaati