cloud (Blog)http://www.turnkeylinux.org/blog/term/81/0
enTurnKey Core 12.0 RC optimized buildshttp://www.turnkeylinux.org/blog/core-rc-squeeze-builds
<p>Last month we announced the <a href="http://www.turnkeylinux.org/blog/core-rc-squeeze">release candidate for TurnKey Core 12.0</a> - the common base for all appliances, based on the rock solid Debian Squeeze (6.0.4).</p>
<p>It took a little longer than expected, but we&#39;ve finally released all the <a href="http://www.turnkeylinux.org/docs/builds">optimized builds</a> for TurnKey Core 12.0RC: <a href="http://www.turnkeylinux.org/core">ISO</a>, <a href="http://www.turnkeylinux.org/docs/builds#vm-default">VMDK</a>, <a href="http://www.turnkeylinux.org/docs/builds#vm-ovf">OVF</a>, <a href="https://hub.turnkeylinux.org">Amazon EC2</a>, <a href="http://www.turnkeylinux.org/docs/builds#openstack">OpenStack</a>, <a href="http://www.turnkeylinux.org/docs/builds#openvz">OpenVZ</a> and <a href="http://www.turnkeylinux.org/docs/builds#xen">Xen</a>.</p>
<p><img alt="Optimized Builds" src="http://www.turnkeylinux.org/files/images/blog/optimized-build-logos.png" style="width: 640px; height: 97px;" /></p>
<p>The optimized builds can be downloaded from the <a href="http://www.turnkeylinux.org/core">core appliance page</a>, directly via the <a href="http://www.turnkeylinux.org/blog/openvz-proxmox">TurnKey channel in ProxmoxVE</a> (OpenVZ), deployed in the Amazon EC2 cloud via the <a href="https://hub.turnkeylinux.org">TurnKey Hub</a>, or via one of the <a href="http://www.turnkeylinux.org/partners">official TurnKey partners</a>&nbsp;(soon).</p>
<h2>
Build specific release notes</h2>
<h3>
Common (ISO)</h3>
<ul>
<li>
See the <a href="http://www.turnkeylinux.org/blog/core-rc-squeeze">announcement</a>.</li>
</ul>
<h3>
Amazon EC2</h3>
<ul>
<li>
<strong>Deployment:</strong> The <a href="https://hub.turnkeylinux.org">TurnKey Hub</a> has been updated to support Core 12.0 deployment and management (Launch new server -&gt; 12.0). Once the full library has been updated to TKL 12.0 it will become the default, and the current release will be moved to Legacy.</li>
</ul>
<h3>
VM optimized (VMDK, OVF)</h3>
<ul>
<li>
<strong>Open-VM-Tools:</strong> Previous VM optimized builds included the proprietry VMWare-Tools, but since VMWare have released a large portion of the code under the GPL, we&#39;ve moved to <a href="http://open-vm-tools.sourceforge.net/faq.php">open-vm-tools</a>.</li>
<li>
<strong>Swap warning:</strong> VMware products might display a warning that no swap space was detected. This is a false positive, as swap is configured in LVM.</li>
</ul>
<h3>
OpenStack</h3>
<ul>
<li>
<strong>Ramdisk required:</strong>&nbsp;TurnKey 12.0 requires the initrd to be registered for successful boot (<a href="http://www.turnkeylinux.org/blog/announcing-openstack-builds#comment-12909">exemplary import and registration code</a>).</li>
</ul>
<h3>
OpenVZ</h3>
<ul>
<li>
<strong>Naming convention:</strong> We&#39;ve updating the naming convention for openvz builds to support vanilla OpenVZ out of the box, and eliminate duplication for the Turnkey PVE channel. Thanks <a href="http://www.turnkeylinux.org/users/jedmeister">Jeremy</a>!</li>
<li>
<strong>Removed NTP daemon:</strong> The NTP daemon has been removed as the clock is managed by the host. Thanks Martin!</li>
<li>
<strong>No more upstart hacks:</strong> Removed Ubuntu upstart hacks as they are not relevant in Debian.</li>
</ul>
<h3>
Xen</h3>
<ul>
<li>
<strong>Xen optimized kernel:</strong>&nbsp;Moved to the Xen optimized kernel provided by Debian (linux-image-xen-686).</li>
</ul>
<p>As always, we need your help in testing the builds. If you come across any issues or have ideas how they can be improved, please post a comment.</p>
http://www.turnkeylinux.org/blog/core-rc-squeeze-builds#commentsappliancescloudcoredebianec2newsopenstackopenvzproxmoxvmwarexenMon, 16 Apr 2012 01:03:17 +0000Alon Swartz3395 at http://www.turnkeylinux.orgNew Hub feature: Server snapshotshttp://www.turnkeylinux.org/blog/hub-snapshots
<p>I usually get excited when adding new features to the <a href="https://hub.turnkeylinux.org">TurnKey Hub</a>. Recent excitement included <a href="http://www.turnkeylinux.org/blog/hub-metrics">server monitoring</a>, <a href="http://www.turnkeylinux.org/blog/reserved-instances">reserved instances</a>, <a href="http://www.turnkeylinux.org/blog/hub-domains">domain management</a>, and the <a href="http://www.turnkeylinux.org/blog/hub-api">Hub API</a>.</p>
<p>I&#39;m very excited about todays annoucement, not only is it awesomely useful, it&#39;s also technically cool!</p>
<h2>
So what are snapshots?</h2>
<p>I&#39;m sure you can guess, but let me explain anyway.</p>
<p>Snapshots can be used with EBS-backed instances to create point-in-time snapshots of the root filesystem, which are persisted to Amazon S3 for storage durability. Snapshots are incremental, meaning that only changes since the last snapshot are saved, taking up less storage, time, and reducing costs (see below for technical details).</p>
<p>Snapshots ask Amazon&#39;s fiber-optic storage backplane to save your server&#39;s disk state while it&#39;s running without impacting performance.</p>
<p>Ok, but what can I do with them?</p>
<h3>
Server clones</h3>
<p>Snapshots can be used as the basis for a new server, essentially creating a clone (the cloud server equivalent of a time machine crossed with a portal to a less obnoxious alternative dimension), for example:</p>
<ul>
<li>
You can clone a production server to create a staging enviroment for testing new features, hacking away, whatever, without the worry of hosing your production server (guess how I tested this new feature).</li>
<li>
You can essentially upgrade your servers hardware if you need the extra horse power, memory or even disk space. Say you were testing an idea with a micro instance, and now its taking off. Firstly congrats, secondly just clone the micro&#39;s latest snapshot to a larger instance size and update the DNS record / re-associate the elastic IP.</li>
<li>
Let you&#39;re imagination run wild!</li>
</ul>
<h3>
EBS Volumes</h3>
<p>Snapshots can be used as a starting point for a new EBS volume, for example:</p>
<ul>
<li>
You mistakenly deleted a file, hosed your database, or whatever bad thing that can happen. You create a volume from the snapshot of your choice, attach it to your instance (which is auto-mounted via <a href="http://www.turnkeylinux.org/blog/ebsmount">ebsmount</a>) and access the data you need.</li>
<li>
Again, let you&#39;re imagination run wild!</li>
</ul>
<h2>
Can I schedule automatic snapshots?</h2>
<p>You sure can! You can schedule automatic zero-load server snapshots for hourly, daily, weekly and monthly frequency, or manually create one at anytime.</p>
<p>There is however a snapshot limit per Amazon account, per region, so when configuring automatic scheduled snapshots, snapshot retention is also configurable to prune old snapshots, keeping you within the limit and saving you money.</p>
<h2>
Sounds cool, what does it look like?</h2>
<p>We&#39;ve added 2 new fields to the server record:</p>
<p><img alt="Snapshots - Server Record" src="http://www.turnkeylinux.org/files/images/blog/snapshots-server.jpg" style="width: 640px; height: 311px; " /></p>
<p>And this is the snapshot dashboard:</p>
<p><img alt="Snapshot - Dashboard" src="http://www.turnkeylinux.org/files/images/blog/snapshots-dashboard.jpg " style="width: 640px; height: 447px; " /></p>
<h2>
Are there any limitations?</h2>
<p>Snapshots only support EBS-backed instances, and not S3-backed instances. This is a technical limitation as snapshots are performed on the EBS-backed root volume, which S3-backed instance do not have.</p>
<p>Snapshots are saved to S3 storage, but they will not appear in your S3 buckets, nor can you access them using the standard S3 API. To access snapshot data you need to create an EBS volume or a server clone.</p>
<p>As mentioned above, there is a limit of the amount of snapshots each Amazon account can have, but you can <a href="http://aws.amazon.com/contact-us/ebs_volume_limit_request/">request to increase your limit</a> (specify you want the snapshots limit increased in the comments.)</p>
<p><strong>Data consistency: Do not solely rely on snapshots for backups, as they may become inconsistent due to disk-buffering and locking. We use <a href="https://hub.turnkeylinux.org/tour/backup/">TKLBAM</a> for our backups, and suggest you do the same.</strong></p>
<h2>
Technical details - snapshots explained</h2>
<p>I mentioned that snapshots are technically cool, and that they are incremental - let me try and explain what that means at how it works behind the scenes.</p>
<p>A snapshot of an EBS volume can be taken at anytime, which asks Amazon&#39;s fiber-optic storage backplane to save the data stored on the volume, at the block level, at that exact point-in-time, to S3 storage.</p>
<p>To improve performance and reduce storage space, Amazon will only copy the blocks of the volume that have changed since your last snapshot - hence incremental.</p>
<p>Now for the extra cool part, unlike regular incremental backup chains, you can delete any previous snapshot. Huh? What? Yep, snapshots are not chained, but are rather conceptually like a table-of-contents of pointers to saved data blocks.</p>
<p>When you delete a snapshot, only the data blocks that are solely used by that specific snapshot are deleted. Data blocks that are used by subsequent snapshots are not. In the below illustration, if SNAP-B is deleted, only SNAP-B:block-2 will be deleted from Amazon S3 as a newer version (SNAP-C:block-2) has already been saved.</p>
<center>
<p><img alt="Snapshots - Blocks" src="http://www.turnkeylinux.org/files/images/blog/snapshots-blocks.jpg" style="width: 600px; height: 476px; " /></p>
</center>
<p><br />
Bottom line, take snapshots for a <a href="https://hub.turnkeylinux.org">spin</a> and let us know what you think.</p>
http://www.turnkeylinux.org/blog/hub-snapshots#commentsawscloudec2hubnewssnapshotsWed, 04 Apr 2012 13:56:02 +0000Alon Swartz3375 at http://www.turnkeylinux.orgNew Hub feature: Cloud server monitoringhttp://www.turnkeylinux.org/blog/hub-metrics
<p>
Ladies and gentle geeks, I&#39;m proud to announce we&#39;ve just pushed out 100%&nbsp;free basic server monitoring to all <a href="https://hub.turnkeylinux.org">TurnKey Hub</a> accounts. This should make it easier to keep tabs on the health and performance of your cloud servers. Existing Hub users don&#39;t need to do anything to enjoy this new feature. It just works.</p>
<h2>
A better server dashboard</h2>
<p>
As you can see in the screenshot below, the server dashboard now includes thumbnail graphs of CPU utilization, disk IO and network traffic for the last hour:</p>
<p>
<img alt="Hub dashboard metric thumbnails" src="http://www.turnkeylinux.org/files/images/blog/metrics-dash.png" style="width: 640px; height: 294px; " /></p>
<!--break-->
<h2>
Give me more!</h2>
<p>
Alright, so instead of the last hour, you want data on how your server was doing last night? Or last week? No problem.&nbsp;CloudWatch samples performance at 5 minute intervals, and stores up to two weeks worth of data.</p>
<meta content="text/html; charset=utf-8" http-equiv="content-type" />
<p>
Clicking on the thumbnail graph pops up a larger interactive graph that lets you zoom in and out, sample performance metrics ondifferent timescales (e.g., hourly, daily, weekly, etc.) and move back and forward in time:</p>
<p>
<img alt="Hub detailed metrics" src="http://www.turnkeylinux.org/files/images/blog/metrics-detailed.png" style="width: 640px; height: 326px; " /></p>
<h2>
No installation, monitoring agents required</h2>
<p>
You don&#39;t need to install or configure any monitoring agents, because the Hub pulls statistics directly from Amazon&#39;s <a href="http://docs.amazonwebservices.com/AmazonCloudWatch/latest/APIReference/">CloudWatch API</a>. CloudWatch in turn gets its data directly from the virtualized hardware layer running underneath your server&#39;s operating system.</p>
http://www.turnkeylinux.org/blog/hub-metrics#commentsawscloudec2hubmonitoringnewsFri, 02 Dec 2011 02:15:11 +0000Alon Swartz2917 at http://www.turnkeylinux.orgTurnKey Hub: Not just adding random featureshttp://www.turnkeylinux.org/blog/hub-not-random-features
<p>
I started writing a review for cloudtask&nbsp;as a comment on the <a href="http://www.turnkeylinux.org/blog/introducing-cloudtask">announcement</a>, but decided it would be better to address a <a href="http://www.turnkeylinux.org/blog/hub-api#comment-8296">topic</a> that was raised by Jeremiah when we launched the <a href="http://www.turnkeylinux.org/blog/hub-api">Hub API</a>:</p>
<blockquote>
<p>
I&#39;m impressed with the way all the pieces of TurnKey just fit together like puzzle pieces. Even as new functionality is added it never seems like something gets patched onto the existing framework. &nbsp;Instead, it just integrates smoothly with what&#39;s already there.</p>
</blockquote>
<p>
Liraz&#39;s <a href="http://www.turnkeylinux.org/blog/hub-api#comment-8302">reply</a>:</p>
<blockquote>
<p>
While the new Hub features are coming out incrementally, they really are part of a bigger unifying vision. I don&#39;t like to hype up vaporware, but we have a lot of great ideas we&#39;re working on and we like to think we&#39;re just getting started. There&#39;s a big gap between what the Hub does right now and what we envision it doing in the future.</p>
<p>
Also note that we&#39;re using the Hub internally to help develop and test TurnKey, so we&#39;re probably using it more than the typical user and run into its limits sooner. We&#39;re scratching our own itch, and the community benefits from that.</p>
</blockquote>
<p>
To recap, some of the features Liraz was referring to in his comment, and some that came after his comment are:</p>
<ul>
<li>
<a href="http://www.turnkeylinux.org/blog/hub-serverstatus-ebs">Hub 1.0 follow-up: improved server status</a></li>
<li>
<a href="http://www.turnkeylinux.org/blog/restore-on-launch">New Hub feature: Auto-Restore TKLBAM backup to a new cloud server</a></li>
<li>
<a href="http://www.turnkeylinux.org/blog/hub-api">Announcing public API for TurnKey Hub</a></li>
<li>
<a href="http://www.turnkeylinux.org/blog/hub-domains">TurnKey Domain management &amp; Dynamic DNS</a></li>
<li>
<a href="http://www.turnkeylinux.org/blog/maintenance-release-11.2">TurnKey 11.2, free micro instances, EBS backed cloud servers</a></li>
<li>
<a href="http://www.turnkeylinux.org/blog/introducing-cloudtask">Introducing CloudTask - a cloud batch execution tool</a></li>
</ul>
<p>
More features are coming, but this a great milestone to stop, reflect, and talk a little about a pain point we&#39;ve had, and how the features we&#39;ve released to this point are helping us solve it.</p>
<h2>
Building and maintaining exponentially growing library</h2>
<p>
TurnKey currently has 45 appliances, available in ISO, VMDK, OVF, EC2 S3 and EBS backed, Xen and Eucalyptus.</p>
<p>
To sum up, thats <strong>585 images</strong> we need to build and maintain.&nbsp;</p>
<p>
The (long overdue) TKL 11 part 2 will double the appliance library. Additionally we are working on supporting more build targets, as well as the new Tokyo EC2 region. Throw 64bit support into the equasion and the number of images to build and maintain will grow to about <strong>3,300</strong>!</p>
<h2>
Automation and a glimpse into a pain point</h2>
<p>
We try to automate everything we do, but even executing and managing the stuff thats already automated needs automation when you reach scale.</p>
<p>
To give you a little glimps into what I mean. When building the Amazon EC2 S3 backed images (in the past), I couldn&#39;t do it locally as my upstream kinda sucks are would take me about 2 weeks to build and upload.</p>
<p>
So do it in the cloud, right? Correct. But building and uploading from an EC2 instance still takes a couple of days, mainly due to subsequent builds and uploading images built for different regions is quiet slow.</p>
<p>
OK, so lets just spawn more servers in the cloud, in the different regions. Yep, this allows us to perform the relative region builds so upload is much faster, and they run in parallel.</p>
<p>
But manually launching each instance from the Hub&#39;s web UI (pre-&nbsp;<a href="http://www.turnkeylinux.org/blog/hub-api">hub-api</a>), updating my /etc/hosts file to keep track of the different instances (pre-&nbsp;<a href="http://www.turnkeylinux.org/blog/hub-domains">hub-domains</a>), logging into each instance and transfering the build infrastructure (pre-&nbsp;<a href="http://www.turnkeylinux.org/blog/restore-on-launch">restore-on-launch</a>), and finally starting the automated build process while making sure nothing breaks is painful to say the least.</p>
<p>
It would also be much faster (and cost the same) if we could split the appliance builds amoungst more instances in each region, but try managing that manually and keep your hair - argh!</p>
<p>
To throw some salt on an already bloody wound, remember we also have to build the other target formats (EBS, VMDK, OVF, etc...). Try do that without going bald.</p>
<h2>
Parallel batch execution with auto-launched cloud servers</h2>
<p>
This is where integrating the features mentioned above, and <a href="http://www.turnkeylinux.org/blog/introducing-cloudtask">cloudtask</a> comes into play.</p>
<p>
Instead of pulling my hair out and waiting days, yesterday I threw a couple of tasks at cloudtask, which launched about 50 instances spanning the globe, our build infrastructure was automatically setup on each instance, and cloudtask started dishing out jobs to each of them.</p>
<p>
Within about 1 hour, I received reports via email that all the builds completed successfully and were published. Pain point gone.</p>
<p>
Now thats automation on steriods! And best of all, I get to keep my hair!<br />
&nbsp;</p>
<p>
As mentioned above, we are just getting started. Lots more to come - it&#39;s getting exciting...</p>
http://www.turnkeylinux.org/blog/hub-not-random-features#commentsawsbatchcloudcloudtaskec2hubparallelFri, 19 Aug 2011 12:50:25 +0000Alon Swartz2595 at http://www.turnkeylinux.orgIntroducing CloudTask - a cloud batch execution toolhttp://www.turnkeylinux.org/blog/introducing-cloudtask
<p>
<em>The cloud</em>. Isn&#39;t that just a new name everyone on the latest hype bandwagon is slapping on the same old stuff? Yes. Or rather, at least the way some clueless marketing types are using it that is. With so much smoke you&#39;d forgive the cynics for thinking there&#39;s no fire. But... there are a few genuinely interesting things an IT guy can do today that just weren&#39;t practical a few years back.</p>
<p>
Like renting an armada of servers you could never afford to buy to parallelize a computational task and get results in an hour instead of days, weeks or even months, for exactly the same cost. Now that&#39;s kind of exciting if you can pull it off.</p>
<p>
We came across a modest version of this problem for various routine TurnKey related maintenance tasks such as rebuilding appliances. It sure was nice to be able to fire up a server on-demand, run a batch job and be able upload new images to sourceforge at 100MB/s. Usually we would leave the batch job running overnight and terminate the server the next day or so. That wasn&#39;t so bad for non-frequent tasks, but we realized we could do better. On Amazon EC2 running 10 servers for 1 hour costs the same as running 1 server for 10 hours. That&#39;s the theory anyhow. In practice launching and controlling many servers by hand can be painful.</p>
<p>
Obviously, a bit of clever automation would be just the ticket. I hate reinventing the wheel so I first tried surveying existing solutions, but I couldn&#39;t find any that fit our needs.</p>
<p>
So I rolled up my sleeves, and about a month or so later cloudtask was born. It&#39;s kind of neat. If you&#39;ve ever had to do this sort of thing by hand or put together an ugly mess of scripts you&#39;ll probably find cloudtask an easier and more reliable primitive to build on.</p>
<p>
Here&#39;s the <a href="http://www.turnkeylinux.org/docs/cloudtask">documentation</a>.</p>
<p>
Here&#39;s a tacky video demo I whipped up (best viewed full screen):</p>
<p>
<embed allowfullscreen="true" allowscriptaccess="always" height="530" src="http://blip.tv/play/g8o%2Bgsv8UgA" type="application/x-shockwave-flash" width="670" wmode="transparent"></embed></p>
http://www.turnkeylinux.org/blog/introducing-cloudtask#commentsawsbatchcloudcloudtaskec2hubnewsparallelFri, 12 Aug 2011 13:17:48 +0000Liraz Siri2585 at http://www.turnkeylinux.orgTurnKey 11.2, free micro instances, EBS backed cloud servershttp://www.turnkeylinux.org/blog/maintenance-release-11.2
<div class="section" id="turnkey-11-2-security-updates-included">
<h2>
TurnKey 11.2: micro instances, EBS support, built-in TurnKey DNS, security updates</h2>
<p>
We just updated the web site and the TurnKey Hub with the new TurnKey 11.2 maintenance release, which includes:</p>
<ol>
<li>
TurnKey Hub support for micro instances, Amazon&#39;s free tier and cloud servers backed by persistent network-attached storage volumes (AKA EBS backed instances).</li>
<li>
Built-in support for TurnKey&#39;s new <a href="http://www.turnkeylinux.org/dns">dynamic DNS</a> service.</li>
<li>
The latest security updates.</li>
</ol>
<p>
<!--break-->
</p>
</div>
<div class="section" id="free-micro-instances">
<h2>
TurnKey Micro instances: 2 cents/hour or 0 cents/hour for a year with the free tier</h2>
We&#39;ve added support for micro instances (613 MB RAM), Amazon EC2&#39;s smallest cloud server type which costs just 2 cents an hour to run, which is less than $15/month if you run a server 24x7. If that isn&#39;t close enough to free for you, Amazon is giving away a year&#39;s worth of micro instance usage to new users as part of their <a href="http://aws.amazon.com/free/">free tier</a> program.
<p>
This means many of you will now be able to try out TurnKey in the cloud free for a year. Yay!</p>
<p>
We would have added support for micro instances as soon as they came out except Amazon designed them to work differently from other instance types we already supported. In particular, we had to add support for EBS backed instances...</p>
</div>
<div class="section" id="support-for-persistent-cloud-servers-that-can-be-turned-on-or-off">
<h2>
EBS backed instances: cloud servers that can be turned off any time</h2>
<p>
Up until now the TurnKey Hub only supported S3 backed instances. These are non-persistent cloud servers with temporary storage that is lost once you destroy the server. This means you can&#39;t just turn off an S3 backed instance to save usage fees when you are not using it, though you could work around this limitation by using <a href="http://www.turnkeylinux.org/tklbam">TKLBAM</a> to backup a cloud server before destroying it and later restoring its state into a new cloud server.</p>
<p>
With the support we&#39;ve added for EBS backed instances, this limitation has been removed. EBS is what Amazon calls its on-demand Network Attached Storage service. The catch is that the Hub has to pre-allocate a fixed size EBS volume for your cloud server to boot from. Unless you are in the free usage tier you&#39;ll have to pay an additional $0.10/GB per month in <a href="http://aws.amazon.com/ec2/pricing/#ebsPricing">EBS storage fees</a> for the convenience (e.g., 50GB EBS volume == extra $5/monthly). The ability to turn off servers when not in use may make up for this extra cost though.</p>
<p>
Speaking of costs, the pricing structure on the TurnKey side is a bit different for EBS backed instances as Amazon doesn&#39;t allow vendors to add a 10% markup to hourly usage fees like we&#39;ve been doing with S3 backed instances. So instead, we&#39;re probably going to be experimenting with a global fixed monthly fee for this feature. After the trial period ends (in a month or so). Currently there is no extra charge.</p>
<p>
Note that this future extra monthly charge will not apply to micro instances.</p>
<p>
<strong>A word of warning</strong>: EBS is not a backup replacement and EBS-backed instances still need to be backed up by TKLBAM. EBS volumes just provides data persistence. It&#39;s a network hard drive that lives in a specific Amazon datacenter. It is not a replacement for backups. For example, if the data on your EBS volume gets accidentally deleted or corrupted, without a backup system to restore from you will be out of luck. TKLBAM on the other hand provides true incremental backups, so good data can&#39;t be accidentally overwritten by bad. Also, TKLBAM uses S3, which is designed by default to provide <a href="http://aws.typepad.com/aws/2010/05/new-amazon-s3-reduced-redundancy-storage-rrs.html">11 nines (99.999999999%)</a> of storage reliability, much higher than EBS.</p>
<h2>
Security updates</h2>
<p>
As most of you know security updates are already <a href="http://www.turnkeylinux.org/docs/automatic-security-updates">installed automatically on first boot and nightly&nbsp;</a> (by default). If you&#39;re using an older version of TurnKey this means you don&#39;t need to do anything to get the latest security fixes. But for new deployments pulling a large number of security updates over the network can take considerable time, so occasional maintenance releases that already include them are a good idea.</p>
<p>
We&#39;re in the process of upgrading our development process so this sort of update will be easier to do in the future and can be done as frequently as necessary.</p>
</div>http://www.turnkeylinux.org/blog/maintenance-release-11.2#commentsappliancesawscloudebsec2hubnewsTue, 02 Aug 2011 12:37:11 +0000Liraz Siri2540 at http://www.turnkeylinux.orgTurnKey Domain management & Dynamic DNShttp://www.turnkeylinux.org/blog/hub-domains
<p>
A while ago I was chatting with Liraz and said <em>&quot;wouldn&#39;t it be great if&nbsp;when launching a cloud server the Hub would perform some magic and&nbsp;assign the server a friendly name? I&#39;m tired of remembering IP&nbsp;addresses, and logging into our DNS management console to setup&nbsp;records.&quot;</em></p>
<p>
Then we thought, <em>&quot;lets make DNS easy, lets make it TurnKey&quot;</em>. So we did...</p>
<p>
No matter your use case, we got you covered:</p>
<h2>
Custom domains</h2>
<p>
Alice uses the Hub to launch and manage her <a href="https://hub.turnkeylinux.org/tour/cloud/">cloud servers</a>. Every time&nbsp;she sets up a new server, she needs to navigate to her DNS management&nbsp;console, log in, go back to the Hub to get the servers IP address,&nbsp;switch back to the DNS console, and setup the appropriate records to&nbsp;point to her server (e.g., <a href="http://www.example.com" title="www.example.com">www.example.com</a> -&gt; 89.231.194.85).</p>
<p>
Not so bad, thats how everyone does it, right? Not anymore!</p>
<p>
We have just released new DNS features in the <a href="https://hub.turnkeylinux.org">TurnKey Hub</a>. Not only can&nbsp;you now manage your DNS settings using a crisp user interface right in the Hub, backed by&nbsp;the awesomeness of <a href="http://aws.amazon.com/route53/">Amazon&#39;s Route53</a> highly available and scalable Domain&nbsp;Name System, but because the Hub also manages your cloud servers, the&nbsp;two systems are tightly integrated.</p>
<p>
<img alt="Hub Domain Management" src="http://www.turnkeylinux.org/files/images/blog/hub-domains1.png" style="width: 640px; height: 463px; " /></p>
<p>
For example, when launching a cloud server, you specify the hostname to&nbsp;associate with your new server, and as soon as your server is running,&nbsp;the DNS records will be automatically created/updated accordingly. You can also associate a hostname with a running server right from the server listing.</p>
<p>
<img alt="Associate domain" src="http://www.turnkeylinux.org/files/images/blog/hub-domains2.png" style="width: 640px; height: 302px; " /></p>
<p>
But wait, there&#39;s more! Don&#39;t want to use Elastic/Static IP&#39;s with&nbsp;your cloud servers? Do you manage a server behind a dynamic IP address?&nbsp;We got you covered - see Dynamic DNS below.</p>
<p>
Are you using the <a href="http://www.turnkeylinux.org/blog/hub-api">Hub API</a> to programmatically launch your servers? Do you&nbsp;use <em>launch another server like this one</em>, or <em>launch this backup in the&nbsp;cloud</em>? We got your covered there as well...</p>
<h2>
TKLAPP.com</h2>
<p>
Bob, unlike Alice, doesn&#39;t own a domain name, so why should he be a&nbsp;second class citizen and not get all these cool new features? He too is&nbsp;tired of remembering IP addresses and sharing them with his friends. He&nbsp;wants an easy to remember name as well.</p>
<p>
Enter TKLAPP.com!&nbsp;TKLAPP.com hostnames and available to all Hub users, and they&#39;re free!&nbsp;Because there is a limited name space, they are available on&nbsp;first-come-first-serve basis, so go grab your own vanity name (or names)&nbsp;before someone else does.</p>
<p>
<img alt="Launch associate domain" src="http://www.turnkeylinux.org/files/images/blog/hub-domains3.png" style="width: 640px; height: 274px; " /></p>
<p>
DNS names aren&#39;t just user friendly, they are sometimes required. For&nbsp;example, appliances which use domain preseeding (such as <a href="http://www.turnkeylinux.org/wordpress">Wordpress</a>,&nbsp;<a href="http://www.turnkeylinux.org/magento">Magento</a>, <a href="http://www.turnkeylinux.org/statusnet">StatusNet</a>, <a href="http://www.turnkeylinux.org/ejabberd">ejabberd</a>) will now be fully configured and ready to&nbsp;rock right off the bat.</p>
<h2>
Dynamic DNS</h2>
<p>
And we didn&#39;t forget about Charlie either, who might be running TurnKey&nbsp;on his own hardware, in a VM or at a hosting provider that supports&nbsp;TurnKey. And given the <a href="http://www.turnkeylinux.org/forum/general/20100930/dyndns-alternative">state of free Dynamic DNS services</a> out there,&nbsp;we created HubDNS.</p>
<p>
<a href="http://www.turnkeylinux.org/docs/hubdns">HubDNS</a>&nbsp;is the TurnKey Dynamic DNS client. It supports both custom&nbsp;domains as well as the free TKLAPP.com domain. It&#39;s also super simple to&nbsp;set up:</p>
<pre>
<code>apt-get update
apt-get install hubdns
hubdns-init HUB_APIKEY foo.tklapp.com
hubdns-update
chmod +x /etc/cron.hourly/hubdns-update # automatic hourly updates</code></pre>
<p>
BTW, HubDNS should work without issues on any Debian/Ubuntu based&nbsp;system. Full installation and usage documentation is available <a href="http://www.turnkeylinux.org/docs/hubdns">here</a>.</p>
<p>
Thoughts, comments, feature requests?</p>
http://www.turnkeylinux.org/blog/hub-domains#commentsawsclouddnsec2hubroute53ubuntuFri, 27 May 2011 09:20:30 +0000Alon Swartz2319 at http://www.turnkeylinux.orgAnnouncing public API for TurnKey Hubhttp://www.turnkeylinux.org/blog/hub-api
<p>
<strong>More power, control, flexibility and automation of cloud servers.</strong></p>
<p>
Alan Kay once said: <em>&quot;Simple things should be simple, complex things should be possible&quot;</em>. We live by those words, and I think we&#39;ve done a pretty good job up until now.</p>
<p>
The <a href="https://hub.turnkeylinux.org">Hub</a> makes launching and managing instances on Amazon EC2 really <a href="https://hub.turnkeylinux.org/tour/cloud/">simple</a>, but the one thing that has been missing is a solution to make complex things possible - i.e., programmatic control.</p>
<p>
Which brings me to todays announcement of the TurnKey Hub API, and HubTools - Python API bindings and CLI tools.</p>
<p>
<strong>Some examples to wet your appetite:</strong></p>
<p>
Launch a new TurnKey Core appliance in the cloud:</p>
<pre>
$ hub-launch core</pre>
<p>
And of course, preseeding is supported, for example:</p>
<pre>
$ hub-launch lamp --db-pass=foobar</pre>
<p>
But wait, there&#39;s more. Lets say you are developing a new Wordpress website in a local VM which is backed up using <a href="https://hub.turnkeylinux.org/tour/backup/">TKLBAM</a>, with a backup ID of 2. Restoring the backup to a new cloud server is as simple as:</p>
<pre>
$ hub-launch 2</pre>
<p>
So how do you know what backups you have available? Which appliances are available and their preseeding options? The status and related information of your cloud servers?</p>
<p>
It&#39;s easy:</p>
<pre>
hub-list-backups
hub-list-appliances
hub-list-servers</pre>
<p>
If the included CLI tools aren&#39;t enough and you need more power, use the Python bindings to develop your own code. It&#39;s really simple.</p>
<p>
For example, lets say I wanted to launch 10 TKL Core servers:</p>
<pre>
<code>for i in range(1, 11):
hub.servers.launch(&quot;core&quot;, label=&quot;TurnKey Core %s&quot; % i)</code>
</pre>
<p>
There is so much you can do with HubTools, it&#39;s only limited to your&nbsp;imagination.</p>
<p>
The full documentation is available <a href="http://www.turnkeylinux.org/docs/hubtools">here</a>. If you don&#39;t have a free Hub&nbsp;account yet, get one <a href="https://hub.turnkeylinux.org">here</a>.</p>
http://www.turnkeylinux.org/blog/hub-api#commentsawscloudec2hubTue, 26 Apr 2011 07:11:05 +0000Alon Swartz2245 at http://www.turnkeylinux.orgNew Hub feature: Auto-Restore TKLBAM backup to a new cloud serverhttp://www.turnkeylinux.org/blog/restore-on-launch
<p>
Since we <a href="http://www.turnkeylinux.org/blog/hub-released">announced</a> the release of <a href="https://hub.turnkeylinux.org/">TurnKey Hub v1.0</a> two weeks ago, we followed up with the <a href="http://www.turnkeylinux.org/blog/hub-serverstatus-ebs">two top issues users reported</a>, and continued to receive awesome feedback - you guys rock, keep it coming!</p>
<p>
A common question we received was related to restoring a backup to a new server. This wasn&#39;t a new question either, last year <a href="http://www.turnkeylinux.org/forum/support/20101106/quickest-way-restore">Phil Bower said</a>: <em>&quot;I only want to run my instances while I&#39;m using them so I&#39;m interested in figuring out the fastest way to create new instances and restore backups to them.&quot;</em></p>
<p>
In light of this, we just released a new Hub feature that streamlines the restore process depending on your use case. It also makes testing your backups even easier!</p>
<h2>
Restore this backup to a new cloud server</h2>
<p>
Alice is a consultant developing a new site for her client on Turnkey Wordpress in a local virtual machine. She is regularly backing up her work with <a href="https://hub.turnkeylinux.org/tour/backup/">TKLBAM</a> and has reached version 1.0 status - time for production.</p>
<p>
Alice logs into her Hub account, goes to the backup dashboard and clicks <em>&quot;Restore this backup to a new cloud server&quot;</em>. The Hub knows which appliance is related to her backup and launches a Wordpress instance. TKLBAM is automatically initialized and her backup is restored, essentially migrating her local VM to the cloud.</p>
<p>
<img alt="Restore on Launch 1" src="http://www.turnkeylinux.org/files/images/blog/restore-on-launch1.png" style="width: 600px; height: 213px; " /></p>
<p>
Alice jumps up and down with excitement. She immediately calls up her client to give them the great news. Development to production in 3 minutes flat!</p>
<h2>
Launch a new server like this one</h2>
<p>
Bob (aka Evil Bob) has a highly customized TurnKey LAMP instance running in the cloud. He has added user accounts, installed packages, saved data on the filesystem as well as MySQL. Additionally, he configured an EBS volume and Elastic IP when he launched the instance. He even customized the firewall rules.</p>
<p>
Bob only needs the instance to be up about for a few days every week while crunching data for his evil plan, but launching a new instance, attaching the EBS, associating the EIP, tweaking the firewall rules and manually restoring his TKLBAM backup is a little tedious.</p>
<p>
Tedious? Not any more! Once a week Bob logs into his Hub account, goes to the server dashboard and clicks &quot;Launch a new server like this one&quot;. &nbsp;He clicks the checkbox to automatically restore his backup, and he&#39;s off! The Hub launches a new server, automatically attaches his EBS volume (<a href="http://www.turnkeylinux.org/blog/ebsmount">EBSMount</a> mounts it), associates his Elastic IP, configures his custom firewall rules and restores his backup.&nbsp;</p>
<p>
<img alt="Restore on launch 2" src="http://www.turnkeylinux.org/files/images/blog/restore-on-launch2.png" style="width: 600px; height: 198px; " /></p>
<p>
The lights dimm, Evil Bob&#39;s face lights up, raises his chin and lets out an evil laugh - MUHAHA!!</p>
<p>
<strong>Note:</strong> The new restore features are not available for passphrase protected backups or legacy builds.</p>
http://www.turnkeylinux.org/blog/restore-on-launch#commentsawsbackupcloudec2hubrestoretklbamThu, 14 Apr 2011 16:19:00 +0000Alon Swartz2212 at http://www.turnkeylinux.orgHub 1.0 follow-up: two top issues users reported + what we're doing about it. http://www.turnkeylinux.org/blog/hub-serverstatus-ebs
<p>
Last week we <a href="http://www.turnkeylinux.org/blog/hub-released">announced</a> the release of <a href="https://hub.turnkeylinux.org">TurnKey Hub v1.0</a>. The response was great, signups went through the roof and many of you went the extra mile and provided detailed feedback on your first impressions from the new version.</p>
<p>
Many thanks to all of you who cared enough to share their experience, whether good or bad. I know it gets thrown around a lot, but we really do care about getting feedback from users. And not just the ego-massaging vanity fair stuff either. <em>&quot;This didn&#39;t do what I expected and it gave me a hard time&quot;</em> is just as valuable if not more so. When you&#39;re so close to something it can be difficult to anticipate the perspective of a new user that may struggle with things we&#39;ve been foolishly taking for granted.</p>
<p>
In the future if you run into any issues, don&#39;t be shy. Speak up, we&#39;re listening. Even if you&#39;re sure someone else must have reported the same issue before you, there&#39;s still value in your report. &nbsp;When users report the same related issues over and over, we take that as a sign of urgency, roll up our sleeves and do something about it.</p>
<p>
For example here are a couple of the most common issues users have been talking to us about since the 1.0 Hub release last week.</p>
<h2>
Issue #1: Improved server status (implemented)</h2>
<h3>
The problem</h3>
<p>
When a new server is launched, its in the <em>pending</em> state (represented by a yellow status icon) while Amazon allocates resources and boots the system. As soon as the server starts booting, Amazon changes the state of the server to <em>running</em> which we&#39;ve been representing as a green status icon.</p>
<p>
Trouble is, a green light sets the expectation that you&#39;re ready to go, but that isn&#39;t necessarily true as the machine may not have finished booting just yet.</p>
<p>
TurnKey <a href="http://www.turnkeylinux.org/docs/automatic-security-updates">automatically installs security updates</a> before exposing potentially vulnerable services to a hostile network, so depending on the number of updates, the boot process can take a few minutes to complete. Meanwhile you&#39;re scratching your head trying to figure out why you can&#39;t connect to the web application in your newly launched cloud server.</p>
<p>
Bottom line, we received multiple reports related to this issue and on closer inspection realized we need to communicate better to the user that they still have to wait, preferably while providing some insight as to what is happening so the user doesn&#39;t lose patience and think the system is &quot;stuck&quot;.</p>
<h3>
The solution</h3>
<p>
TurnKey cloud servers now update the Hub with their boot status, which the Hub server dashboard displays and periodically updates. Most importantly, the Hub only gives you a green light and changes the status to running when the server has fully finished booting.</p>
<p>
<img alt="Hub server status" src="http://www.turnkeylinux.org/files/images/blog/hub-server-status.png" style="width: 600px; height: 97px; " /></p>
<p>
The server dashboard uses AJAX to auto-update when the server status changes.</p>
<h2>
Issue #2: AWS free tier, and EBS backed instances (request for feedback)</h2>
<p>
Not long after Amazon announced support for micro instances, the AWS free tier plan was born, with much fanfare and excitement. Micro instances are especially small servers that don&#39;t pack a lot of punch for production use, but can none-the-less be useful in some low-end usage scenarios.</p>
<p>
We were initially hoping to support micro instances immediately in the Hub, but micro instances presented a few unique technical challenges due to extra limitations imposed by Amazon. For example, they had to be EBS-backed, which means they&#39;re booted from a persistent virtual network drive known as an EBS volume in AWS lingo.</p>
<p>
In case you&#39;re wondering, EBS backed instances are different from the regular temporary storage instances in that they provide the ability to stop and start instances at any time. You don&#39;t pay for CPU hours while an EBS backed instance is &quot;off&quot;, just the persistent storage space.</p>
<p>
Alas, as the Hub doesn&#39;t yet support setting up EBS based servers, TurnKey users couldn&#39;t launch micro instances or take advantage of the free tier, which we understand is a bit disappointing.</p>
<p>
Since the 1.0 release we&#39;ve received an increasing number of requests to add support for micro instances, the AWS free tier and EBS backed instances to the Hub.</p>
<p>
We realize it&#39;s important to some users, but we&#39;re still trying to figure out if it&#39;s important enough to override other priorities. To get more information we&#39;ve set up 2 community survey polls (<a href="http://www.turnkeylinux.org/polls/hub-micro-instances">micro/free-tier</a>, <a href="http://www.turnkeylinux.org/polls/hub-ebs-instances">ebs-backed</a>) and are inviting users to register their vote.<br />
&nbsp;</p>
<p>
Finally, if you have any other feedback on how we can improve the Hub, don&#39;t hesitate to let us know!</p>
http://www.turnkeylinux.org/blog/hub-serverstatus-ebs#commentsawscloudec2hubMon, 04 Apr 2011 08:16:56 +0000Alon Swartz2187 at http://www.turnkeylinux.orgAnnouncing TurnKey Hub v1.0 - now officially out of private betahttp://www.turnkeylinux.org/blog/hub-released
<p style="text-align: center; padding-top: 10px;">
<a href="https://hub.turnkeylinux.org"><img alt="Hub Front" src="http://www.turnkeylinux.org/files/images/blog/hubv1-front.png" style="width: 640px; height: 461px; " /></a></p>
<p>
When we first announced the <a href="http://www.turnkeylinux.org/blog/hub-private-beta">TurnKey Hub private beta</a> about 9 months ago, we had limited capacity (invitation only) and a modest feature set. Since then we tested, bugfixed, removed bottlenecks and added features, constantly improving the Hub with the help and feedback from our excellent beta users. Thank you so much!</p>
<p>
With the release of <a href="http://www.turnkeylinux.org/blog/turnkeylinux-11-part1">TurnKey 11</a> which was tightly integrated with TKLBAM and the Hub, the amount of Hub invitation requests exploded. We were prepared for this and managed to scale the Hub smoothly without any serious issues.</p>
<p>
With several months of testing, feedback and bugfixes under our belt we are now confident enough to officially announce, a bit earlier than planned, that the Hub is out of private beta. As of today, the <a href="https://hub.turnkeylinux.org">Hub</a> is open to all, and new users will no longer be required to request an invitation.</p>
<p>
Existing users can rest easy though. We will continue to carefully monitor the Hub&#39;s performance. There should be no interruptions to the service. Worse case scenario, if we start hitting unforseen capacity issues we will temprarily reintroduce the limit on new signups.</p>
<h2>
Review of notable changes since the initial release</h2>
<p>
<strong>TurnKey Backup and Migration</strong></p>
<ul>
<li>
A few months into the private beta we announced support for <a href="https://hub.turnkeylinux.org/tour/backup
">TurnKey Backup and Migration</a> (AKA TKLBAM), which amongst other uses makes previously difficult tasks such as <a href="http://www.turnkeylinux.org/blog/backups-are-hard
">testing your backups</a> much easier.</li>
<li>
In response to demand, we&#39;ve added support for configurable backup retention. Users can specify how many full backups they would like to keep for any given server backup (set to unlimited by default).</li>
</ul>
<p>
<strong>TurnKey Cloud Servers</strong></p>
<ul>
<li>
Support for TurnKey Linux 11 images (legacy images still available to ease migration).</li>
<li>
Basic pre-launch configuration: No more having to fiddle with the <a href="http://www.turnkeylinux.org/blog/end-to-default-passwords">default passwords</a> after an instance launches. The Hub supports <a href="http://www.turnkeylinux.org/blog/hub-preseeding">pre-seeding appliance configuration before launch</a>. This makes up for not having console access that would usually be required for first boot configuration.</li>
<li>
TKLBAM pre-initialization: No more having to cut and paste your Hub APIKEY to initialize TKLBAM. The Hub pre-initializes TKLBAM automatically when the instance is first launched.</li>
<li>
Upgradeable Kernels: We&#39;ve figured out how to make it easy to update the kernel via pv-grub.</li>
<li>
Preset launch region automatically chosen by geo-location of user.</li>
</ul>
<p>
<strong>General stuff</strong></p>
<ul>
<li>
Performance optimizations, improved stability and error handling.</li>
<li>
Refined the look and feel with an update to the theme.</li>
<li>
We now try harder to explain how the Hub works and what it&#39;s good for before and after you sign up. For example we&#39;ve added nice visual tours of the <a href="https://hub.turnkeylinux.org/tour/backup">Backup and migration</a> and <a href="https://hub.turnkeylinux.org/tour/cloud">Cloud servers</a> features.</li>
<li>
We&#39;ve added a <a href="https://hub.turnkeylinux.org/pricing
">pricing page</a> answering frequently asked questions. Yes, the Hub is still free. You pay Amazon directly for the cloud resources you use.</li>
<li>
Improved start page to get you going once you sign up. Once you setup your account, this transforms into a dashboard that provides a high level overview and quick access links.</li>
<li>
New and improved notifications (<a href="http://www.turnkeylinux.org/blog/django-notifications">growl style</a>).</li>
<li>
Removed invitation requirement and added support for OpenID signup and authentication.</li>
<li>
Added functionality to change account email.</li>
<li>
Full internationalization support (<a href="http://www.turnkeylinux.org/blog/postgresql-latin1-utf8">UTF-8</a>).</li>
<li>
APT archive geo-location API service for choosing the <a href="http://www.turnkeylinux.org/blog/auto-apt-archive">closest package archive</a>.</li>
<li>
Link to <a href="http://www.turnkeylinux.org/blog/privacy-policy">Privacy policy</a>.</li>
</ul>
<p>
As usual, feedback is appreciated. If you don&#39;t have a TurnKey Hub account yet, go get one now or try out the demo. If you already have a TurnKey Hub account, go check out the new stuff.</p>
<p>
The TurnKey Hub lives at: <strong><a href="https://hub.turnkeylinux.org">https://hub.turnkeylinux.org</a></strong></p>
http://www.turnkeylinux.org/blog/hub-released#commentsawsbackupcloudec2hubnewstklbamWed, 30 Mar 2011 12:42:48 +0000Alon Swartz2166 at http://www.turnkeylinux.orgTurnKey Linux 11 released (part one)http://www.turnkeylinux.org/blog/turnkeylinux-11-part1
<p>
Ladies and gentlemen, part 1 of the TurnKey Linux 11 release is now officially out, including 45 new images based on Ubuntu 10.04.1. We pushed out the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1">11.0 release candidates</a> 3 months ago, and with the help of the community have tested the images and resolved the few remaining issues.</p>
<!--break-->
<p>
Part 1 mostly refreshes the existing roster of appliances in the library. In the upcoming part 2 we&#39;ll release the new appliances the community has been helping us develop over the last year. This will roughly double the size of the library.</p>
<p>
A handful of new appliances have also been squeezed in:</p>
<table align="center" border="0" style="border: 0pt none; text-align: center;">
<tbody>
<tr>
<td>
<div>
<a href="http://www.turnkeylinux.org/joomla"> <img src="http://www.turnkeylinux.org/files/images/icons/joomla.jpg" /></a></div>
<a href="http://www.turnkeylinux.org/joomla">Joomla 1.6</a></td>
<td>
<div>
<a href="http://www.turnkeylinux.org/magento"> <img src="http://www.turnkeylinux.org/files/images/icons/magento.jpg" /></a></div>
<a href="http://www.turnkeylinux.org/magento">Magento</a></td>
<td>
<div>
<a href="http://www.turnkeylinux.org/statusnet"> <img src="http://www.turnkeylinux.org/files/images/icons/statusnet.jpg" /></a></div>
<a href="http://www.turnkeylinux.org/statusnet">StatusNet</a></td>
<td>
<div>
<a href="http://www.turnkeylinux.org/prestashop"> <img src="http://www.turnkeylinux.org/files/images/icons/prestashop.jpg" /></a></div>
<a href="http://www.turnkeylinux.org/prestashop">PrestaShop</a></td>
<td>
<div>
<a href="http://www.turnkeylinux.org/vtiger"> <img src="http://www.turnkeylinux.org/files/images/icons/vtiger.jpg" /></a></div>
<a href="http://www.turnkeylinux.org/vtiger">vTiger CRM</a></td>
</tr>
</tbody>
</table>
<div class="section" id="what-s-changed-since-the-release-candidates">
<p>
This was mostly a side effect of the original (misguided) plan to do one big massive release with over 80 appliances.</p>
<h2>
What&#39;s changed since the release candidates</h2>
<ul>
<li>
<p class="first">
<strong>VM optimized builds</strong>: are now available, in two exciting flavors...</p>
<ol class="arabic simple">
<li>
<a href="http://www.turnkeylinux.org/docs/builds#vm-default">Default</a>: this is primary downloadable VM build. Works best with VirtualBox, low-end VMWare products (Player, Workstation, Server).</li>
<li>
<a href="http://www.turnkeylinux.org/docs/builds#vm-ovf">OVF</a>: OVF is the new VM standard supported by VirtualBox and high-end VMWare products (e.g., ESX, vSphere).</li>
</ol>
<p>
We&#39;re also working on providing images optimized for other popular virtualization platforms such as Xen, UEC / Eucalyptus and OpenVZ.</p>
</li>
<li>
<p class="first">
<strong>Default passwords</strong>: You no longer have to keep track of any default passwords or change them after installation. TurnKey now <a href="http://www.turnkeylinux.org/blog/end-to-default-passwords">helps you configure them on first boot</a>, via the console.</p>
<p>
For headless deployments without a console, it&#39;s possible to <a href="http://www.turnkeylinux.org/docs/inithooks">pre-seed</a> answers to first boot configuration questions.</p>
</li>
<li>
<p class="first">
<strong>/etc under git</strong>: Automatic revision control of /etc using <a href="https://blueprints.launchpad.net/turnkeylinux/+spec/add-etckeeper-package">etckeeper</a>, as suggested by Jeremiah Snapp. If a configuration change you made breaks something, just roll it back!</p>
</li>
<li>
<p class="first">
<strong>LVM snapshots</strong>: <a href="https://blueprints.launchpad.net/turnkeylinux/+spec/use-lvm-snapshots-with-tklbam">Fixed LVM snapshots</a> by adding 10% unallocated disk space to default LVM configuration. This will make it easier to add support for atomic filesystem backups in upcoming versions of TKLBAM.</p>
</li>
<li>
<p class="first">
<strong>Amazon EC2 / TurnKey Hub related changes</strong>:</p>
<ul>
<li>
<p class="first">
<em>Hub does TKL 11</em>: <a href="https://hub.turnkeylinux.org">TurnKey Hub</a> now deploys TurnKey Linux 11 images by default, though support for older legacy images is still available to ease migration.</p>
</li>
<li>
<p class="first">
<em>Basic pre-launch configuration</em>: No more having to fiddle with the default passwords after an instance launches. TurnKey Hub now supports <a href="http://www.turnkeylinux.org/blog/hub-preseeding">pre-seeding appliance configuration before launch</a>. This makes up for not having console access that would usually be required for first boot configuration.</p>
</li>
<li>
<p class="first">
<em>TKLBAM pre-initialization</em>: No more having to cut and paste your Hub APIKEY to initialize TKLBAM. The TurnKey Hub pre-initializes TKLBAM automatically when the instance is first launched.</p>
</li>
<li>
<p class="first">
<em>Upgradeable Kernels</em>: We&#39;ve figured out how to make it easy update the kernel.</p>
<p>
Kernel upgrades were previously not supported on Amazon EC2, because each Amazon Machine Image (AMI) had to be associated with a specific Amazon Kernel Image (AKI). Now instead of associating the image to a specific kernel, we associate it with a special EC2 compatible bootloader (<a href="http://wiki.xensource.com/xenwiki/PvGrub">pv-grub</a>), which can bootstrap whatever kernel is configured from within the system (e.g., security fix).</p>
</li>
</ul>
</li>
<li>
<p class="first">
<strong>Updated website documentation</strong></p>
<ul class="simple">
<li>
All 45 appliance pages (e.g., <a href="http://www.turnkeylinux.org/core">TurnKey Core</a>) now reflect the latest image versions.</li>
<li>
Documented <a href="http://www.turnkeylinux.org/docs/builds">appliance build types</a>.</li>
<li>
Updated <a href="http://www.turnkeylinux.org/docs/installation-appliances-virtualbox">virtualbox installation tutorial</a>, <a href="http://www.turnkeylinux.org/docs/virtualization">virtualization notes</a>, <a href="http://www.turnkeylinux.org/docs/automatic-security-updates">automatic security updates</a> pages.</li>
</ul>
</li>
<li>
<p class="first">
<strong>Bug fixes</strong>: An assortment of fixes for issues reported by the community.</p>
</li>
</ul>
<p>
See the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1">previous announcement of the release candidates</a> for changes since the last <a href="http://www.turnkeylinux.org/blog/maintenance-release">maintenance release</a> (2009.10-2, based on Ubuntu 8.04 LTS).</p>
</div>
<div class="section" id="what-s-next">
<h2>
Many, many thanks to...</h2>
<ul>
<li>
Everyone who helped test the release candidates and provided ideas and feedback.</li>
<li>
The many rivers of upstream: Ubuntu, Debian and all of the wonderful open source communities who give love and write code for the software that goes into TurnKey.</li>
<li>
<a href="http://www.turnkeylinux.org/users/jedmeister">JedMeister</a>, <a href="http://www.turnkeylinux.org/users/adrianmoya">Adrian Moya</a>, <a href="http://www.turnkeylinux.org/users/basilkurian">Basil Kurian</a>, and <a href="http://www.turnkeylinux.org/users/rikgoldman">Rik Goldman</a> - pillars of the TurnKey community who have inspired us with their dedication and generosity.</li>
<li>
TurnKey enthusiasts everywhere. Without you, TurnKey&#39;s audience, there really wouldn&#39;t be a point!</li>
</ul>
<h2>
What&#39;s next</h2>
<ul class="simple">
<li>
Images optimized for Xen, Eucalyptus / UEC and OpenVZ</li>
<li>
Part 2: double the size of the library</li>
<li>
64-bit support</li>
<li>
Debian squeeze based beta builds (we&#39;ve decided to skip Lenny)</li>
<li>
PostgreSQL support for TKLBAM</li>
<li>
TurnKey Hub stuff
<ul>
<li>
Support for Amazon EBS backed root devices</li>
<li>
Support for micro instances. Amazon&#39;s 1-year free tier will now allow free evaluation of TurnKey on Amazon EC2.</li>
<li>
Support for larger instances (64-bit support required).</li>
<li>
Support for more hosting providers.</li>
</ul>
</li>
</ul>
</div>http://www.turnkeylinux.org/blog/turnkeylinux-11-part1#commentsappliancescloudec2hubnewsubuntuThu, 27 Jan 2011 11:18:10 +0000Liraz Siri1922 at http://www.turnkeylinux.orgSecure, flexible and scalable Amazon EC2 instance preseedinghttp://www.turnkeylinux.org/blog/hub-preseeding
<p>
I&#39;d like to introduce Joe. He is a good looking, experienced sys-admin and like all good sysadmins, he has more stuff to do than time to do it.</p>
<p>
Joe wants to get up and running on Amazon EC2 with a Wordpress&nbsp;installation, and chooses to do so with a pre-configured appliance.&nbsp;These are the steps Joe performs:</p>
<ul>
<li>
Joe logs into his favourite Amazon EC2 console, specifies a&nbsp;Wordpress appliance, and other configurations.</li>
<li>
Clicks launch.</li>
<li>
Once the instance is running, he logs in using his SSH public key&nbsp;and changes the root password (it was set randomly on firstboot, right?).</li>
<li>
He then proceeds to change the MySQL root password as well (also&nbsp;set randomly on firstboot, hopefully!). Joe knows how to do this&nbsp;as he&#39;s an experienced sys-admin, do you?</li>
<li>
Finally, Joe logs into Wordpress using the default admin password&nbsp;(he noted the default password in the release notes before launching), resets the password and specifies his own email for the&nbsp;account.</li>
</ul>
<p>
While performing the above, Joe was holding his breath and working as&nbsp;fast as he could because he was previously hit by a <a href="http://en.wikipedia.org/wiki/Botnet">botnet</a> looking for&nbsp;random systems using default passwords and was compromised. Luckily this&nbsp;time he came out unscaved.</p>
<p>
Does this sound familiar? Well, it should because that&#39;s how it&#39;s&nbsp;<em>mostly</em> being done.</p>
<p>
You might be thinking to yourself <em>&quot;but I used the TurnKey Hub to set the&nbsp;root password for my instances, which also set the database password&quot;</em>.&nbsp;True, that has been a feature of the Hub from day one, but with the&nbsp;release of TurnKey 11.0 and the <a href="http://www.turnkeylinux.org/blog/end-to-default-passwords">end to default passwords</a>, we&#39;ve&nbsp;extended the Hub to support preseeding as well.</p>
<p>
The idea behind this was not only to make cloud deployments more secure,&nbsp;but to make it much easier. We wanted to simplify the process for Joe&nbsp;from the above to this:</p>
<ul>
<li>
Joe logs into the <a href="https://hub.turnkeylinux.org">Hub</a>, selects Wordpress and preseeds the&nbsp;configuration.</li>
<li>
Clicks launch.</li>
</ul>
<p class="rtecenter">
<img alt="" src="http://www.turnkeylinux.org/files/images/blog/preseed_hub.jpg" style="width: 480px; height: 528px;" /></p>
<div id="cke_pastebin">
The above is not a mock-up of a future implementation, it&#39;s live on the <a href="https://hub.turnkeylinux.org">Hub</a>.</div>
<div>
So how does it work? Read on...</div>
<div id="cke_pastebin">
&nbsp;</div>
<h2>
Brainstorming a solution</h2>
<div id="cke_pastebin">
The problem in preseeding an instance is sending the information&nbsp;(securely) to the instance.</div>
<div>
So how do you do it?</div>
<div id="cke_pastebin">
&nbsp;</div>
<h3>
Idea #1: pass it through Amazon EC2 user-data?</h3>
<div id="cke_pastebin">
If you know a little about Amazon EC2 you&#39;ll know that when launching an&nbsp;instance you can specify <a href="http://www.turnkeylinux.org/blog/ec2-userdata">user-data</a>&nbsp;which is accessible from the&nbsp;instance via Amazon&#39;s API.</div>
<div>
&nbsp;</div>
<div>
But wait, do you really want to store authentication credentials in&nbsp;user-data?&nbsp;</div>
<div id="cke_pastebin">
&nbsp;</div>
<div id="cke_pastebin">
You could, but because any process on the instance that can open a&nbsp;network socket can access the user-data as it never expires, you&#39;ll&nbsp;probably want to firewall off the Amazon API as soon as it&#39;s not&nbsp;required anymore during instance initialization. But maybe the user of&nbsp;the instance needs access to the Amazon API? Crippling the service by&nbsp;design isn&#39;t a good solution in my honest opinion.</div>
<div id="cke_pastebin">
&nbsp;</div>
<h3>
Idea #2: store it in the Hub&#39;s database, and let the server query the API</h3>
<div id="cke_pastebin">
So, instead of sending authentication credentials via user-data, why&nbsp;not send a unique identifier (e.g., SERVER_ID), so the instance can use&nbsp;the Hub&#39;s API to pull the credentials?</div>
<div id="cke_pastebin">
&nbsp;</div>
<div id="cke_pastebin">
Well, you could, but that would mean the Hub service needs to&nbsp;store the instance&#39;s configuration, passwords and all, in its database and delete it when it&#39;s no longer needed. Storing an item in a database for just one use is inelegant. But it&#39;s a natural solution if you only have a database, as I dicussed in a previous <a href="http://www.turnkeylinux.org/blog/django-celery-rabbitmq">blog post</a>, <em>&quot;when all you&nbsp;</em><em>have is a hammer, everything looks like a nail&quot;</em>.</div>
<div>
&nbsp;</div>
<div>
In my opinion, it ultimately comes down to <a href="http://en.wikipedia.org/wiki/Separation_of_concerns">separation of concerns</a>. For this type of pattern, the most natural solution would be some sort of&nbsp;messaging service. The Hub publishes a message to a&nbsp;queue, which the instance consumes.</div>
<div id="cke_pastebin">
&nbsp;</div>
<h3>
Idea #3: pass it as messages using the Advanced Message Queuing Protocol (AMQP)</h3>
<div id="cke_pastebin">
So whats wrong with messaging? Nothing really, so long as you take care&nbsp;when designing the system for confidentiality and integrity - we don&#39;t&nbsp;want others eavesdropping on messages, or sending spoofed messages.</div>
<div id="cke_pastebin">
&nbsp;</div>
<div id="cke_pastebin">
Messages that fail a CRC or cannot be decrypted successfully should be&nbsp;discarded, and removed from the queue so not to block it.</div>
<div id="cke_pastebin">
&nbsp;</div>
<h2>
Designing infrastructure that is secure, scalable and extendible</h2>
<div id="cke_pastebin">
The solution we came up with is designed to be secure, scalable and extendible. Eventually it will support other cloud&nbsp;hosting providers, as well as provide bi-directional secure&nbsp;communication for future Hub-based services still under development.</div>
<div id="cke_pastebin">
&nbsp;</div>
<div id="cke_pastebin">
The solution uses each of the <em>brainstormed</em>&nbsp;solutions above for what they&nbsp;were designed for, and no more.</div>
<div id="cke_pastebin">
&nbsp;</div>
<div class="rtecenter" id="cke_pastebin">
<img alt="" src="http://www.turnkeylinux.org/files/images/blog/preseed_dfd.jpg" style="width: 640px; height: 412px;" /></div>
<div id="cke_pastebin">
<span class="Apple-style-span" style="color: rgb(90, 51, 32); font-family: 'Trebuchet MS',Trebuchet,'Nimbus Sans L',sans-serif; line-height: 16px; font-size: 13px; font-weight: bold; letter-spacing: 1px;">Data Flow Diagram (DFD) explained:</span></div>
<ol>
<li>
The user specifies preseeding data.</li>
<li>
The Hub tells Amazon EC2 to launch the instance with user-data which&nbsp;includes the SERVER_ID.</li>
<li>
The Hub creates a direct message exchange and queue for the server, which is configured to only receive messages sent from the Hub.</li>
<li>
The Hub publishes <a href="http://www.turnkeylinux.org/blog/python-symmetric-encryption">symmetrically encrypted</a> messages (incl. a CRC) to the&nbsp;server queue with preseeding data that only the server can&nbsp;decrypt.</li>
<li>
The instance pulls user-data from the Amazon EC2 API (SERVER_ID).</li>
<li>
The Instance registers itself with the Hub via an SSL secured API using the&nbsp;SERVER_ID, which responds back with the server subkey and messaging&nbsp;secret. Note that this can only be done once for security.
<ul>
<li>
<div id="cke_pastebin">
<strong>subkey</strong>: A one way hash generated from the user&#39;s APIKEY. It is&nbsp;unique for each server registered in the Hub, and is used as&nbsp;part of the exchange and queue naming structure.</div>
</li>
<li>
<div>
<strong>secret</strong>: A secure unique hash used for message encryption and&nbsp;decryption.</div>
</li>
</ul>
</li>
<li>
The instance consumes messages from the queue. Messages are decrypted&nbsp;and passed to the callback for processing (preseeding messages appends the <em>arg=value</em>&nbsp;to inithooks.conf).</li>
<li>
During inithooks execution, inithooks.conf is sourced so preseeding&nbsp;will happen. Once inithooks.conf is no longer needed, it is&nbsp;deleted.</li>
</ol>
<p>
In addition to authentication related preseeding, <a href="http://www.turnkeylinux.org/docs/tklbam">TKLBAM</a> is also&nbsp;preseeded with the HUB_APIKEY and is initialized, so performing the&nbsp;initial backup is as easy as executing <em>tklbam-backup</em> or using the TKLBAM webmin module.</p>
<p>
As always, the client side code that implements the above is open&nbsp;source, and can be found in the following projects: <a href="http://code.turnkeylinux.org/hubclient">hubclient</a>, <a href="http://code.turnkeylinux.org/tklamq">tklamq</a>, <a href="http://code.turnkeylinux.org/inithooks">inithooks</a>, as well as the above&nbsp;mentioned blog posts.</p>
<p>
<strong>Take the <a href="https://hub.turnkeylinux.org">Hub</a> for a spin and let us know what you think.</strong></p>
http://www.turnkeylinux.org/blog/hub-preseeding#commentsamazonawscloudec2hubsecurityMon, 03 Jan 2011 08:43:25 +0000Alon Swartz1829 at http://www.turnkeylinux.orgFinding the closest APT package archive using GeoIP and indexinghttp://www.turnkeylinux.org/blog/auto-apt-archive
<p>
In preparation for TurnKey&#39;s upcoming release based on Ubuntu Lucid 10.04 LTS,&nbsp;we are knocking off todo list items. One of them is code-named&nbsp;<a href="http://code.turnkeylinux.org/auto-apt-archive">auto-apt-archive</a>. As you can guess from its name, the objective is to&nbsp;configure the closest APT package archive mirror, automatically, without&nbsp;user intervention. It does this by leveraging a new GeoIP service&nbsp;provided by the <a href="https://hub.turnkeylinux.org">TurnKey Hub</a>.</p>
<p>
By using the closest archive, it is usually much faster, will lessen&nbsp;the load on Ubuntu&#39;s main package archive which has been the default up&nbsp;until now, and in certain circumstances, cheaper (for example, bandwidth within Amazon EC2 regions is free).</p>
<p>
BTW, TurnKey EC2 builds already include a similar optimization, which&nbsp;leverages <a href="http://www.turnkeylinux.org/blog/amazon-ec2-metadata">ec2metadata</a> to get its associated region and construct the URL for the region specific Ubuntu APT archives.</p>
<p>
The new auto-apt-archive solution will replace the old Amazon EC2 adhoc&nbsp;solution, but will also be included in all TurnKey builds, whether it&nbsp;be bare-metal, virtual machines, VPS&#39;s or cloud deployment.</p>
<p>
<span class="Apple-style-span" style="color: rgb(90, 51, 32); font-family: 'Trebuchet MS', Trebuchet, 'Nimbus Sans L', sans-serif; line-height: 25px; font-size: 20px; font-weight: bold; letter-spacing: 1px; ">So how does it work</span></p>
<p>
Firstly, you might recall a post I made last month, with the somewhat&nbsp;similar title <a href="http://www.turnkeylinux.org/blog/geoip-amazon-regions">Finding the closest data center using GeoIP and&nbsp;indexing</a>. The GeoIP implementation<span class="Apple-style-span" style="line-height: 21px; ">&nbsp;details are similar, so I won&#39;t repeat them here.</span></p>
<p>
For those interested in how auto-apt-archive works, it goes something like this:</p>
<p>
On firstboot, auto-apt-archive is called by an <a href="http://code.turnkeylinux.org/inithooks">inithook</a>, which contacts the Hub&nbsp;requesting the closest&nbsp;<a href="https://launchpad.net/ubuntu/+archivemirrors">Ubuntu APT package archive</a>, and updates APT sources&nbsp;lists accordingly.</p>
<p>
The Hub looks up the requesting IP address using GeoIP to find the associated country code which is used in the archive URL.</p>
<p>
Ubuntu have implemented a wildcard domain configuration for the archive&nbsp;mirrors, making the URL construction really simple. In the case that there&nbsp;is no local APT archive in your country, you will be routed to Ubuntu&#39;s&nbsp;main package archive. When one does become available, you&#39;ll&nbsp;automatically be routed there.</p>
<pre>
http://$CC.archive.ubuntu.com/ubuntu
</pre>
<div id="cke_pastebin">
What about Amazon EC2 you ask? Well, the Hub checks if the IP address is&nbsp;associated with an Amazon EC2 instance it launched, and if it does, returns the&nbsp;region specific archive URL.</div>
<pre>
http://$REGION.archive.ubuntu.com/ubuntu</pre>
<div id="cke_pastebin">
In the future, when we add more Cloud deployment options to the Hub which&nbsp;have local APT package archives, they will be automatically supported as&nbsp;well.</div>
<div>
&nbsp;&nbsp;</div>
<div id="cke_pastebin">
And lastly, don&#39;t forget that Debian appliances are in the works, so&nbsp;<a href="http://www.debian.org/mirror/list">Debian APT package archives</a> are also supported. Debian haven&#39;t&nbsp;implemented wildcard DNS, so the Hub looks up the best archive in an&nbsp;index (similar to the amazon region indexes), and returns the archive URL.</div>
<pre>
http://ftp.$CC.debian.org/debian</pre>
<div>
&nbsp;</div>
<div id="cke_pastebin">
Just as with the previous geoip/index post, we need your help to tweak the indexes and mapping logic,&nbsp;as you have better knowledge and experience on your connection latency,&nbsp;and mirror speed. If you think we should associate your country/state to&nbsp;a different archive, please let us know.</div>
http://www.turnkeylinux.org/blog/auto-apt-archive#commentsclouddebianec2hububuntuMon, 04 Oct 2010 12:26:30 +0000Alon Swartz1592 at http://www.turnkeylinux.orgTKLBAM: a new kind of smart backup/restore system that just workshttp://www.turnkeylinux.org/blog/announcing-tklbam
<h2>
<a name="drumroll"></a>Drum roll please...</h2>
<p>
Today, I&#39;m proud to officially unveil TKLBAM (AKA TurnKey Linux Backup and Migration): the easiest, most powerful system-level backup anyone has ever seen. Skeptical? I would be too. But if you read all the way through you&#39;ll see I&#39;m not exaggerating and I have <a href="#screencast">the screencast</a> to prove it. Aha!</p>
<p>
<img alt="" src="http://www.turnkeylinux.org/files/images/blog/tklbam-shot1_650.jpg" style="border: 1px solid rgb(170, 170, 170); margin-left: 9px; width: 650px; height: 317px;" /></p>
<p>
This was the missing piece of the puzzle that has been holding up the Ubuntu Lucid based release batch. You&#39;ll soon understand why and hopefully agree it was worth the wait.</p>
<h2>
<a name="design"></a>We set out to design the ideal backup system</h2>
<p>
Imagine the ideal backup system. That&#39;s what we did.</p>
<h3>
Pain free</h3>
<p>
A fully automated backup and restore system with no pain. That you wouldn&#39;t need to configure. That just magically knows what to backup and, just as importantly, what NOT to backup, to create super efficient, encrypted backups of changes to files, databases, package management state, even users and groups.</p>
<h3>
Migrate anywhere</h3>
<p>
An automated backup/restore system so powerful it would double as <em>a migration mechanism</em> to move or copy fully working systems anywhere in minutes instead of hours or days of error prone, frustrating manual labor.</p>
<p>
It would be so easy you would, shockingly enough, actually <a href="http://www.turnkeylinux.org/blog/backups-are-hard">test your backups</a>. No more excuses. As frequently as you know you should be, avoiding unpleasant surprises at the worst possible timing.</p>
<p>
One turn-key tool, simple and generic enough that you could just as easily use it to migrate a system:</p>
<ul class="simple">
<li>
from Ubuntu Hardy to Ubuntu Lucid (get it now?)</li>
<li>
from a local deployment, to a cloud server</li>
<li>
from a cloud server to any VPS</li>
<li>
from a virtual machine to bare metal</li>
<li>
from Ubuntu to Debian</li>
<li>
from 32-bit to 64-bit</li>
</ul>
<h3>
System smart</h3>
<p>
Of course, you can&#39;t do that with a conventional backup. It&#39;s too dumb. You need a vertically integrated backup that has system level awareness. That knows, for example, which configuration files you changed and which you didn&#39;t touch since installation. That can leverage the package management system to get appropriate versions of system binaries from package repositories instead of wasting backup space.</p>
<p>
This backup tool would be smart enough to protect you from all the small paper-cuts that conspire to make restoring an ad-hoc backup such a nightmare. It would transparently handle technical stuff you&#39;d rather not think about like fixing ownership and permission issues in the restored filesystem after merging users and groups from the backed up system.</p>
<h3>
Ninja secure, dummy proof</h3>
<p>
It would be a tool you could trust to always encrypt your data. But it would still allow you to choose how much convenience you&#39;re willing to trade off for security.</p>
<p>
If data stealing ninjas keep you up at night, you could enable strong cryptographic passphrase protection for your encryption key that includes <a href="http://www.turnkeylinux.org/blog/tklbam-backup-passphrase">special countermeasures against dictionary attacks</a>. But since your backup&#39;s worst enemy is probably staring you in the mirror, it would need to allow you to <a href="http://www.turnkeylinux.org/node/1533">create an escrow key</a> to store in a safe place in case you ever forget your super-duper passphrase.</p>
<p>
On the other hand, nobody wants excessive security measures forced down their throats when they don&#39;t need them and in that case, the ideal tool would be designed to optimize for convenience. Your data would still be encrypted, but the key management stuff would happen transparently.</p>
<h3>
Ultra data durability</h3>
<p>
By default, your AES encrypted backup volumes would be uploaded to inexpensive, ultra-durable cloud storage designed to provide <a href="http://aws.typepad.com/aws/2010/05/new-amazon-s3-reduced-redundancy-storage-rrs.html">%99.999999999</a> durability. To put 11 nines of reliability in perspective, if you stored 10,000 backup volumes you could expect to lose a single volume once every 10 million years.</p>
<p>
For maximum network performance, you would be routed automatically to the cloud storage datacenter <a href="http://www.turnkeylinux.org/blog/geoip-amazon-regions">closest to you</a>.</p>
<h3>
Open source goodness</h3>
<p>
Naturally, the ideal backup system would be open source. You don&#39;t have to care about free software ideology to appreciate the advantages. As far as I&#39;m concerned any code running on my servers doing something as critical as encrypted backups should be available for peer review and modification. No proprietary secret sauce. No pacts with a cloudy devil that expects you to give away your freedom, nay worse, your data, in exchange for a little bit of vendor-lock-in-flavored convenience.</p>
<h3>
Tall order huh?</h3>
<p>
All of this and more is what we set out to accomplish with TKLBAM. But this is not our wild eyed vision for a future backup system. We took our ideal and we made it work. In fact, we&#39;ve been experimenting with increasingly sophisticated prototypes for a few months now, privately eating our own dog food, working out the kinks. This stuff is complex so there may be a few rough spots left, but the foundation should be stable by now.</p>
<h2>
Seeing is believing: a simple usage example</h2>
<p>
We have two installations of TurnKey Drupal6:</p>
<ol class="arabic simple">
<li>
Alpha, a virtual machine on my local laptop. I&#39;ve been using it to develop the TurnKey Linux web site.</li>
<li>
Beta, an EC2 instance I just launched from the TurnKey Hub.</li>
</ol>
<p>
In the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1">new TurnKey Linux 11.0</a> appliances, TKLBAM comes pre-installed. With older versions you&#39;ll need to install it first:</p>
<pre>
apt-get update
apt-get install tklbam webmin-tklbam
</pre>
<p>
You&#39;ll also need to link TKLBAM to your TurnKey Hub account by providing the API-KEY. You can do that via the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1#webmin-tklbam">new Webmin module</a>, or on the command line:</p>
<pre class="literal-block">
tklbam-init QPINK3GD7HHT3A
</pre>
<p>
I now log into Alpha&#39;s command line as root (e.g., via the console, SSH or web shell) and do the following:</p>
<pre class="literal-block">
tklbam-backup
</pre>
<p>
It&#39;s <em>that</em> simple. Unless you want to change defaults, no arguments or additional configuration required.</p>
<p>
When the backup is done a new backup record will show up in my Hub account:</p>
<p>
<img alt="" src="http://www.turnkeylinux.org/files/images/blog/tklbam-shot1_650.jpg" style="border: 1px solid rgb(170, 170, 170); margin-left: 9px; width: 650px; height: 317px;" /></p>
<p>
To restore I log into Beta and do this:</p>
<pre class="literal-block">
tklbam-restore 1
</pre>
<p>
That&#39;s it! To see it in action watch the video below or better yet log into your <a href="https://hub.turnkeylinux.org">TurnKey Hub</a> account and try it for yourself.</p>
<h2>
<a name="screencast"></a>Quick screencast (2 minutes)</h2>
<p>
<iframe width="670" height="530" src="https://www.youtube.com/embed/BML9g5kaIvg" frameborder="0" allowfullscreen></iframe>
</p>
<p>
Best viewed full-screen. Having problems with playback? Try the <a href="http://www.youtube.com/watch?v=BML9g5kaIvg">YouTube</a> version.<br />
<br />
The screencast shows TKLBAM command line usage, but users who dislike the command line can now do everything from the comfort of their web browser, thanks to the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1#webmin-tklbam">new Webmin module</a>.</p>
<h2>
Getting started</h2>
<p>
TKLBAM&#39;s front-end interface is provided by the <a href="https://hub.turnkeylinux.org/">TurnKey Hub</a>, an Amazon-powered cloud backup and server deployment web service currently in private beta.</p>
<p>
If you don&#39;t have a Hub account already, <a href="https://hub.turnkeylinux.org/account/invite/request/">request an invitation</a>. We&#39;ll do our best to grant them as fast as we can scale capacity on a first come, first served basis. <strong>Update: </strong>currently we&#39;re doing ok in terms of capacity so we&#39;re granting invitation requests within the hour.</p>
<p>
To get started log into your Hub account and follow the basic usage instructions. For more detail, <a href="http://www.turnkeylinux.org/docs/tklbam">see the documentation</a>.</p>
<p>
Feel free to ask any questions in the comments below. But you&#39;ll probably want to check with <a href="http://www.turnkeylinux.org/docs/tklbam/faq">the FAQ</a> first to see if they&#39;ve already been answered.</p>
<h2>
Upcoming features</h2>
<ul class="simple">
<li>
<strong>PostgreSQL support</strong>: PostgreSQL support is in development but currently only MySQL is supported. That means TKLBAM doesn&#39;t yet work on the three PostgreSQL based TurnKey appliances (PostgreSQL, LAPP, and OpenBravo).</li>
<li>
<strong>Built-in integration</strong>: TKLBAM will be included by default in all future versions of TurnKey appliances. In the future when you launch a cloud server from the Hub it will be ready for action immediately. No installation or initialization necessary.</li>
<li>
<strong>Webmin integration</strong>: we realize not everyone is comfortable with the command line, so we&#39;re going to look into developing a custom webmin module for TKLBAM. <strong>Update</strong>: we&#39;ve added the <a href="http://www.turnkeylinux.org/blog/turnkey-11.0rc-part1#webmin-tklbam">new TKLBAM webmin module</a> to the 11.0 RC images based on Lucid. In older images, the webmin-tklbam package can also be installed via the package manager.</li>
</ul>
<h2>
Special salute to the TurnKey community</h2>
<p>
First, many thanks to the brave souls who tested TKLBAM and provided feedback even before we officially announced it. Remember, with enough eyeballs all bugs are shallow, so if you come across anything else, don&#39;t rely on someone else to report it. Speak up!</p>
<p>
Also, as usual during a development cycle we haven&#39;t been able to spend as much time on the community forums as we&#39;d like. Many thanks to everyone who helped keep the community alive and kicking in our relative absence.</p>
<p>
Remember, if the TurnKey community has helped you, try to pay it forward when you can by helping others.</p>
<p>
Finally, I&#39;d like to give extra special thanks to three key individuals that have gone above and beyond in their contributions to the community.</p>
<p>
By alphabetical order:</p>
<ul class="simple">
<li>
<a href="http://www.turnkeylinux.org/users/adrianmoya">Adrian Moya</a>: for developing appliances that rival some of our best work.</li>
<li>
<a href="http://www.turnkeylinux.org/user/19534">Basil Kurian</a>: for storming through appliance development at a rate I can barely keep up with.</li>
<li>
<a href="http://www.turnkeylinux.org/users/jedmeister">JedMeister</a>: for continuing to lead as our most helpful and tireless community member for nearly a year and a half now. This guy is a frigging one man support army.</li>
</ul>
<p>
Also special thanks to Bob Marley, the legend who&#39;s been inspiring us as of late to keep jamming till the sun was shining. :)</p>
<h2>
Final thoughts</h2>
<p>
TKLBAM is a major milestone for TurnKey. We&#39;re very excited to finally unveil it to the world. It&#39;s actually been a not-so-secret part of our vision from the start. A chance to show how TurnKey can innovate beyond just bundling off the shelf components.</p>
<p>
With TKLBAM out of the way we can now focus on pushing out the next release batch of Lucid based appliances. Thanks to the amazing work done by our star TKLPatch developers, we&#39;ll be able to significantly expand our library so by the next release we&#39;ll be showcasing even more of the world&#39;s best open source software. Stir It Up!</p>
http://www.turnkeylinux.org/blog/announcing-tklbam#commentsamazonbackupcloudhubnewss3tklbamubuntuWed, 08 Sep 2010 13:32:47 +0000Liraz Siri1542 at http://www.turnkeylinux.org