Tag: AWS Lambda

Building on last month’s post, for a fixed amount of time, I’m making various API methods discussed on our website freely available. For example, you can GET arxiv figure data using the following curl, where of course the query string must correspond to data in our database: curl -k ‘https://plot2txt-staging.us-east-1.elasticbeanstalk.com/record?type=figure&source=astro&label=temp’ Here we’re requesting figures fromContinue Reading free api methods part 1

Some time ago I launched a little project, mining data from arxiv; you can read about it in other blog posts. Specifically, I modeled figures from about 500k figures as Gaussian mixture models, in order to create some features, so figures might be ultimately represented as graphs for comparison. More ordinary methods might suffice tooContinue Reading arxiv mining

We’re edging closer to officially releasing available API methods, including a core OCR method (text-lines) that allows for text extraction in the presence of extraneous objects like embedded images and so forth. Image up/download times combined with computation cost at the backend amounts to several seconds, which isn’t too bad. Using curl to POST data,Continue Reading Mining text from document pages

One of the distinct advantages of devOps philosophy as implemented using AWS is the ability to rapidly kick out a new service. The new template-match API is a good example. Knowing the algorithm, I was able to quickly pull something together from the p2t C++ library, an example input image attached. The algorithm determines whereContinue Reading DevOps with AWS

I’ve uploaded about 500k of the 1M or so figures extracted from arxiv, to AWS S3 for storage. As mentioned, each figure is represented as a Gaussian mixture model (in *CSV) and an image showing the locations (indices) of the model components. Below is a quick summary of the dataset. The largest three subject areasContinue Reading Figure Search Engine III

I’ve spent some time coming to grips with nodejs, in order to develop truly functional lambda functions. Thinking asynchronously (something nodejs is adept at) is a little challenging if you’re not accustomed to it. While the AWS examples are very useful, the process of uploading and triggering through the console doesn’t make for quick turnContinue Reading Lambdas: Epilogue

Ok; we have our ducks in a row, so let’s fire off the test event from the CLI and see what happens (it probably goes without saying at this point, but make sure all your AWS objects are consistent with regards to zone. Here, we use us-east-1): $ aws lambda invoke –invocation-type Event –function-name CreateThumbnailContinue Reading Lambda Functions Part 4

Allegedly we’ve laid the groundwork, let’s dive in and see if we can create the Lambda function that will work with S3 containers, as described in the last post. From the CLI, this is a simple matter of executing the following: $ aws lambda create-function –region us-east-1 –function-name CreateThumbnail –zip-file fileb://textTN.zip –role arn:aws:iam::485603662400:role/lambda_s3_exec_role –handler CreateThumbnail.handlerContinue Reading Lambda Functions Part 3

In order to learn the ropes a little more, I decided to try a worked Lambda example. Recall that last time the build was adjusted to suit amazon EC2, so now I need to learn a little more about the infrastructure, in order to deploy and run in a real scenario. I’m anticipating that theContinue Reading Lambda Functions Part 2

I’ve embarked on the journey to move the p2t backend to AWS Lambda, as intimated in the previous post. In order to leverage AWS Lambda, the leap to Node.js or Java must be made. Thus on this glorious 4th of July weekend I jumped in with both feet. For a native binary to work, wrappedContinue Reading Lambda Functions Part 1