As a developer who deals daily with RESTful interfaces at his job, I found this to be rather intuitive. It may be complicated but would you be so kind as to elaborate on what is unnecessarily complicated about this interface? You might think "Oh, you're just moving data around" but add on top of that security like SSL support, scalability, namespaces and the ability to store very large hundred GB objects then... Yeah, the end result is going to be a bit more than PUT <Data>Object</Data>. It's well documented as far as I can tell. I haven't used it so I don't know if this documentation is worthless but it looks comprehensive at first glance.

So, theodp, if you were a developer you would look at this and see a set of interfaces to web services done in a RESTful manner. You would say, "Oh, my users want to use Google storage but they need more of a drag and drop interface." Then you would spend a couple weeks using Ruby on Rails and Scriptaculous to make virtual folders or buckets or whatever your application calls them and using the elegance of RoR with the UI of Scriptaculous so the user can move their photos or data from your server to the cloud or vice versa. You could really use anything you want to interact with it but I would bet these two GPL compatible tools would result in the most rapid of web application development.

So three sentences with links to Google besmirching them for being smart will get you on the frontpage of Slashdot these days? Really the substance of the 'story' here is essentially "WTF?! So complicated it must Suck!"

Offering the kind of 'user-friendly' API...

Here's a final hint: API stands for Application Programming Interface is not supposed to be user-friendly. It's supposed to be developer-friendly. I hope I don't sound like a Google fanboy but this is a nontrivial task and I would defend the API they have produced. The documentation is far more than you would get from a CS PhD. You want me to take notice of your mindless drivel, theodp? Get off your ass, code an interface for this API and then point out how the API and documentation is lacking in a step by step post. That would be helpful and deserve a place in Slashdot's programming section. What you have here is not.

Slashdot is so non-technical these days it's a complete fucking joke. Ignorance just spews on anything even remotely related to software development. Please drop "news from nerds" from the slogan. Replace with "lip service for sycophants".

It isn't slashdot that has become non-nerdy, it's that being a nerd has become "cool", very unlike it was when slashdot started. These days, anybody who knows that you make your computer stop by clicking "start" thinks (s)he's a nerd, even if they couldn't copy a file without a GUI, let alone have ever heard of Linux or BSD or any other non-Microsoft OS (which these days actually have GUIs).

In the old days, a submission like this most likely wouldn't have been posted, but now we have the firehose, where every nerd wannabe can vote a story up. There are still very good, technical stories here (there was one a couple of days ago about mathematics) -- you just have to ignore the ones like this one voted up by the wannabes. That said, I haven't looked at Google's APIs.

God, I never thought I'd see the day when we would be considered cool! Just laugh smugly and enjoy being cool instead of being a wannabe.

That said, sometimes I say stupid things here (probably a lot this week, I've had the flu and it's affected my mental faculties).

Which is why, to this day, it's so much easier to get a GUI up and running in other OS's other than Windows.

While most other OS's will 'just work' getting Windows to display the GUI can be an involved and frustrating task requiring the modification of.conf files or running command line based configuration tools.

Slashdot is so non-technical these days it's a complete fucking joke. Ignorance just spews on anything even remotely related to software development. Please drop "news from nerds" from the slogan. Replace with "lip service for sycophants".

/agreed

People would rather talk about eCiggerettes than eLectronics these days on slashdot.

It's a straight forward documented restful api. No biggie, written a few myself and it is always a bonus to get some decent usage examples.

I'm not sure you would be classified as a google fanboy for pointing out the obvious.

Looking over the API and the simpler nature of the subject I doubt it would take a few weeks. If you have some code lingering around to manage similar API's you can sling together an app over a weekend.

This guy is a bit of an idiot. There's a ridiculously simple Python wrapper right here [google.com], based on the Boto S3 library, that Google has already built/customized for him if he doesn't feel like writing his own REST wrapper in the language of his choice.

They provide examples there in a few lines of code each for uploading, copying files, reading metadata, deleting files, etc.

I'm not getting this either. It don't look too hard for a developer to me. If you are a user, what could be easier? With Gmail you now have drag and drop to attach a file with HTML5 support. There is getting to be developers doing all kinds of cool stuff with the tools provided and they have anything you could want. They change stuff really fast but that's not really a complaint.

Granted, this story is grandstanding. But still, this is what you have to do to copy from the article:

"Create source and destination URIs."

"Create new destination URI with the source object name as the destination object name." (clone_replace_name)

"Create a new destination key object."

"Retrieve the source key and create a source key object."

"Create a temporary file to hold our copy operation."

"Copy the file."

That seems like a lot of steps, and a couple of them seem very strange to me, namely the clone_replace_name.

I agree that complex tasks require complex APIs. I just don't see why this is such a complex task. We're not using SSL, namespaces or storing a gigantic file here, and I don't see any reason why those features should make the process that much harder. If you want to store large data in the cloud, why should it be so much harder than storing data on a regular filesystem? You don't have "namespaces" on the filesystem, just folders and they just work. SSL "just works." Large files are not intrinsically different from small files. There aren't any ACLs in this example. Where's the complexity? Shouldn't simple things be simple?

The answer is because the cloud is ultimately about marketing and selling expensive crap to enterprises that don't need it, so a burdensome API is just another way of making things that should be cheap more expensive. Expensive developers up on their marketing will get to charge 5x as much because it will take them 5x as much work to do simple things. "Everyone wins."

There's also the consideration that, if you are designing an API that you hope will see broad 3rd-party uptake, you are probably better off making it comprehensive and internally sensible and consistent, rather than starting small and hoping that the additions you tack on in the future don't become a perverse mess.

If it turns out, after the fact, that 90% of your users just want easy access to 10% of it, it isn't exactly impossible to have a trivial_subset library that sits on top, and makes it easy for

I think this is a terrible idea. It violates the principle of "make it as simple as possible, but not any simpler". Some things are just complicated. Even if the UI is nice and clean, what it interacts with is not and the users have to know about that. I don't think any UI could be understood by a 5 year old and I'm fine with that.

ALL of it is user facing. That's the very point of an API. The user is the developer.

This is a very, very important concept. As I said in my other post, this is a good API, a usable API. But so many APIs aren't usable. API usability should ALWAYS be considered when releasing a public library/service.

Sure. But you shouldn't be able to explain Photoshop or vi to a 5 year old, either.

Sure I can. Photoshop lets you paint on pictures, and Vi is like a piece of paper that you can write on.

With an API the difference is that you should be able to assume that your user will have a common lower bound on their knowledge. If your API deals with multi-threading, to be effective you probably need to assume your user knows the fundamentals of multi-threaded programming. Or, at least that the user has some base level of knowledge in computer science.

Because, letters in the alphabet are "a code" but we don't refer to them as "codes for wording".So, everyone who is familiar with "the code" used IS supposed to understand it. Regardless if we are talking alphabet, kanji, C++ or Python.

Unless it is poorly coded and/or a mess - which is a part of the reason why the OP questions the current practice.

So are English, German, Spanish, Chinese, Japanese, Navajo, Greek, Hebrew, Arabic... and how many people can write in more than a couple of those? How many Americans can write in more than one of them? For that matter, how many Americans can write correctly, legibly, and coherently in their own native language?

My point is, it takes some work to learn a new language (or code). Not everyone can do it, and not everyone wants to.

well yeah, the code would probably look similar to the one on a turing machine with a binary tape. You can have N operations on M pieces of data and every time just ask something like "run operation number 0? y/n" "run opeartion number 1? y/n"... then "run it on 0 pieces of data? y/n"... then "for parameter 1 use piece number 0? y/n"... etc

The only nonintuitive thing is the name "bucket", which might be better called "zone" or "filesystem". Other than that, it looks like it provides just about what I'd expect of a high-level filesystem representation.

Sheesh, just think about what the complaints would be if they provided something closer to VFS-type mappings so people ended up commonly rewriting half of FUSE to get their data where they like.

I assume they used the word "bucket," because that's what Amazon calls the same abstraction in S3. And that's really the target audience for this storage system: developers who are currently using Amazon S3 or are familiar with it. The idiot complaining about this API is angry that he went to a hardware store and didn't find the pipes already in the shape of his sprinkler system, possibly because he doesn't know what a hardware store is.

The only nonintuitive thing is the name "bucket", which might be better called "zone" or "filesystem".

It might be better to call it "bucket", if one of your biggest target audiences was, say, developers already using and familiar with Amazon S3, a popular existing service in the same space that calls the same thing a "bucket" rather than a "zone" or "filesystem".

Since when does an API need to be user friendly? I found Google's documentation much more user friendly and straightforward than say Microsoft's.NET documentation on File I/O. It's not an end-user product. Just skimming over the contents of the linked sites, it seems very easy to use even if you're not an advanced programmer. If you don't understand what's on those websites after some thorough reading, please hand in your geek card.

What exactly is hard to use or cryptic about a RESTful API? If such a thing strains your brain too much you probably are in the wrong line of work. I'd recommend you get a job flipping burgers but even that may be way more than your intellectually capable of.

This isn't to discredit the idea of ease of use or good design - god knows Google graphs requires way more hoops than it should (compare, say, Visifire).

I think it's easy to look at the developer's guide and just flee in terror, but honestly if that's your reaction, Google storage API is probably not the droid you're looking for. If you need simple file sharing that a typical user can appreciate without having to read a manual, Dropbox may be more appropriate; Google Storage API is written with developers in mind.. I'm a big fan of some of Google's APIs, Dropbox, and Google Docs for sure.

I was expecting something really crazy and complex but what I saw was well documented and made sense. Seriously, how on earth is this front page news on slashdot?? I wont repeat the many well made statements that "API's arent for users" above. I'm just surprised this has made it to the front page as a developers link. I sure hope I don't work with the sub. at any point if he thinks this is an example of people being "too smart for their own good"./saddened

InteroperabilityGoogle Storage is interoperable with a large number of cloud storage tools and libraries that work with services such as Amazon Simple Storage Service (Amazon S3) and Eucalyptus Systems, Inc.

Basiacally, google is essentially building on what has become an industry standard for cloud storage.

Which, btw, is awesome. If you can poll via an API the current cost of storage at various providers, and transfer costs are cheap, you can move your data across the world between providers effortlessly (same API and all that Jazz).

If this a storage system to be used as a filesystem, why does it need a API? Write a OS filesystem driver then everything can use it. Easy enough to do in userspace with FUSE on Linux/BSD/OSX and Dokan on Windows. Everything is a filesystem, and this really seams to be a filesystem, so make it a filesystem. But maybe I'm missing something here.....

Yes you are. This is not a "storage system to be used as a filesystem" it's an implementation of the Amazon S3 interface that provides remote, redundant key/value storage (where the value in this case is a bucket of bytes). There's nothing to stop you implementing a file system on top of it; but the API provided by Google is at a lower level than that. Which is a good thing as a standard file system is not necessarily the best way to use this kind of storage.

The “user” of an API is an application programmer. The “user” of a UI in general is your grandmother.

Both of them have to be of just the right complexity... neither unnecessarily complex, nor overly simplistic. If the interface is unnecessarily complex, it will be harder to use; if it is overly simple, it may not be usable at all.

Both of them have to be usable, but an application programmer’s idea of “usable” will be very different from your grandmother’s. Equ

Seriously, Google has a number of products with extremely simple, user friendly interfaces. Their search engine, you know, the reason that they are who they are, the reason that anyone knows about them, is a prime example. Type in what you want, it finds what you need. No special syntax needed, no complex logical operations to try and get results, just key in terms or phrases and you get good results.

No special syntax needed, no complex logical operations to try and get results, just key in terms or phrases and you get good results.

Haven't we been doing this since the early 90s?

Hotbot
yahoo
askjeeves
altavista...

It's because you have more control no your searches ('filetype:jpg',...) with relevant results and hat made google what it is, is "unintrusive and related advertizing". It used to be about the "most flashy", "most blinky", "largest", "impossible to click away" and what have you.

Things become much more complicated then first impression when you try to really explain something. For example I went to a football game with a group of Chinese grad students and they asked me how a team can score points. I thought to myself this is easy, and began to explain the rules.

1. Touchdowns are worth 7 points... err they are worth 6 points technically2. After a touchdown the scoring team can decide to kick the ball through the uprights for 1 pointOr3. The scoring team can decide to run another regular play and if they enter the end-zone again on that 1 play they get 2 points.4. Fields goals are 3 points and are scored when the team on offense can kick the ball through the uprights.5. The defense can score points if they can tackle an offensive player in the end-zone while they are holding the football. The defensive team then gets 2 points and gets the ball kicked to them on the following play instead of the normal system where the scoring team kicks the ball to the other team.6. If the defense can steal the ball and run into the end-zone they are facing then it is a touchdown and rule 2 and 3 apply.

By the end of this discussion they were more confused then when we started. So when you say how hard can it be to explain how to store a file questions like.

1. How to delete?2. How to rename?3. How to create folders or other organizational structures?4. How to move items between organizational structures?5. How to copy an item already in storage?6. How to download multiple files?7. Can security be set or changed?8. Oh yeah and how to I upload a file in the first place?

The more precision you apply to a discussion the more complicated they tend to get. Just like a touchdown is 7 points is easier to understand, upload a file is easy too.

After skimming the file-copying code, I agree with the people
who say it's not complicated. I'm not a Python programmer either.
The example functions they gave look like good starting points for
wrappers that would provide the higher level, "get, send, delete"
sort of functionality the poster wants. The only thing that confuses
me is why you have to have "config = boto.config" when the config
variable isn't used in the rest of the code. To me, it looks like
you're only interested in the side effects of retrieving the configuration
and not the result. Couldn't you just "boto.config()" or something at
program startup? Of course
that's probably more of a Python question from somebody who is ony passably familiar
with the language. It's nothing complicated about the API.

seems logical if slightly redundant. So we need some objects instead of plaintext names...

dst_key_name = dst_uri.object_name + os.sep + src_uri.object_name...wtf... oh, we are trying to create a local filename... that's some convoluted way to do it.

new_dst_uri = dst_uri.clone_replace_name(dst_key_name)

err... so our local disk file needs to be placed at... "pets/dogs/poodle.jpg". Now that's some way to get there!

dst_key = new_dst_uri.new_key()

oh, that was just the NAME of the new key... so we need the actual "key"... again, what for?

src_key = src_uri.get_key()

again, if we have the URI object, why do we need some "key" object? Isn't the dedicated URI object good enough?

tmp = tempfile.TemporaryFile()
src_key.get_file(tmp)
tmp.seek(0)...can't we read directly from a file instead of creating temporary one? So the "uri" of a file is not good enough, the "key" of a file is not good enough, we need a "tempfile" object extra?

dst_key.set_contents_from_file(tmp)

I recommend a reading about a hammer factory factory factory [joelonsoftware.com]. This one doesn't overdo factories, just abstraction layers. I can spot four: filename (string), URI, key, file handle within the key. WHO needs that???

This looks like a nice low-level API for doing really interesting and complicated things. Unfortunately, they neglected to include a high-level API to deal with what will be by far the most common use cases. Sure, it's not so difficult to implement an upload_file(filepointer, uri) function with this, but given the huge proportion of developers using this library that are going to need exactly this sort of function, do we really need all of them reinventing the wheel?

Powerful and complex functionality is good, but the most common use cases got that way for a reason. Specifically accounting for them, even if only through a set of basic frontend functions, brings major productivity boosts to the programmers that use your library. It is a thing worth doing, and it sounds like the Google folks neglected to do that in this case.

Google has just put the best holiday logo ever on the homepage. A working PacMan!
I wonder what's the worldwide economic impact of this joke, in terms of lost productivity. It probably beats most terrorist acts and natural disasters:).

Last time I checked, the Google search page was pretty straightforward. For more complicated systems, get set up on Google Voice. I'm so used to phone-tree hell that I was pretty stunned with how quick and painless that was. They are happy to put user experience as a primary concern when they're trying to acquire users. For a storage API, frankly having granddad want to try it out might not be in their interests.

I know I’m (sadly) a minority here. And I know that I will probably get modded to into oblivion. But except from the stupid overengineering... come on!How about for a change actually learning something, when it is useful for you?

* I’m not even really saying that people are too dumb. It’s just that most people grew up in a culture, where it made more sense, to complain and feel entitled, to getting spoon-fed, than to understand it themselves. Where intelligent people get hate, and dumb people get special treatment (e.g. it not being allowed to point out that fact about their mental performance).So naturally, they choose the more efficient way.

But the thing is, that we all are very much capable of grasping those complex concepts that we always say we were too dumb for. It’s just an excuse. And the more it is used, the more mental growth we miss. So after some time, we really have a hard time using our brains. Just like with a muscle. Just like we all are born with the ability to some day run for hours, every day, in the heat.

So, no, they are not too smart. We’re just used to being lazy as hell.

After reading through the API, if anything, it's too simple. You can't copy a bucket without reading it from Google's servers and writing it back, which is far slower than a copy carried out within their high-speed network. The "list" capability isn't well documented. The security model is about as dumb as the UNIX/Linux one; it doesn't have capabilities or anything like that. Bucket transactions are themselves atomic, but there are no user-specified atomic transactions. You can't, for example, rename "current" to "old" and "new" to "current" as an atomic transaction. (That's a normal operation in SQL, and a useful one when you've constructed a new copy of a mostly-static table and want to make it live.) Nor do buckets have version management. There's no way to read replication status; although bucket data is supposedly replicated, when does this happen? Right after uploading a bucket, or some time later?

It may seem a bit hilarious, apparently this kind of crap (like having bucket names conform to DNS) happens when you want to use web services as your OS. Not too hard if you just implement this once.

This bit is just silly, good for giggles but are they serious about requiring zone editing to expose a database table? Nooooo....

For example, let's assume your domain is example.com and you want to make travel maps available to your customers. You could create a bucket in Google Storage called travel-maps.example.com, and then create a CNAME record in DNS that redirects requests from travel-maps.example.com to the Google Storage URI. To do this, you publish the following CNAME record in DNS

I didn't quite catch how you copy data to other domains, since it looks like you use a gs:// prefix to reach google storage but you say gs://cats and it is still in your account not at google's root server.. kind of annoying though maybe there's a way around it?

I think the 1024 byte limit is totally bogus, that's pretty short if it has to hold the URI path through your virtually nested buckets. Although I've seen Windows flake out at 255 character paths.. That and the bit about a "flat hierarchy", which is an oxymoron, and how you can't nest buckets but you can do so "virtually" by putting slashes in your bucket names, as if it isn't just a normal URI, they're just joshing you, a little bit of fun y'know. "Bare metal" indeed, more like stripping the metaphor down to bare CGI.

It is funny you have to allocate your own temporary file as a buffer for uploading a file, though of course that's what happens in Perl CGI. Which then makes you wonder why you cannot set a max upload data size for your app.. Of course the GSUtil command line tool looks pretty simple.

Otherwise, Animats' post is to right to the point. It isn't really that great. Kind of a bare minimum is more like it. And they stick with REST... so you should hope for a nearby library to exist that will save you not have to start implementing wierd HTTP verbs.. you have to really want this as implementing it seems as much fun as pulling teeth slowly.

I wonder if Google Storage can be abused as a way to host phishing pages?

There's a phishing page [phishtank.com] that's been on Google Sites since February. Google is good about kicking off most phishing pages, but this one is different. Here's the phishing page as a web page. [google.com] The actual hostile page (which is a bogus login page for Stickam) is on the "Click here to download your attachment". The actual url is http://2699962600425641406-a-1802744773732722657-s-sites.googlegroups.com/site/stickamcomlogindo/login.html?attachauth=ANoY7cpc6fembideFQyYULstnVDU-XMkgwzNLFkUv77Suh8bUq_LGrFRQ-RtLkw6pEPJb5Vk0XW4JMbOVQtqT_R6CjNCh5N2r29quoFkE5Cq1XQXUFhuegVtr4kQUMN9T3dT3yO1q-FthiahDl45UqMmFfD6gKSYwQP4bsgVoM-N5cQN0hHRvDZskuvmTdy0lqnQqUhmKFYP&attredirects=0. That's probably a page in Google Storage.

This raises the question of whether Google should be running hostile-code checks on publicly-accessible Google Storage pages.

UI != API. A proper API doesn't need to be simple, it needs to work properly and consistently. Not to get too subtle, but a complex API for something like this is perfectly fine as long as it is not overly complicated, if you get what I mean.

4. Sufficiently engineered3. Over-engineered2. Under-engineered1. Doesn't work or works on accident.

That is to say, average developers tend to nail the common case, but lack the experience or knowledge to spot the corner cases. Your "above average" developer wants to demonstrate his knowledge by optimizing for as many corner cases as possible at the expense of simplicity in the common case. The well-above average developer can balance the common