You are here: Home / Past Courses / Spring 2012 - ECPE 293A / Projects / AWS Developer Tools

AWS Developer Tools

Here are some useful Amazon Web Services Developer Tools:

AWS Management Console - Web Interface

To access, use your private link (for your IAM account) as stored in the Sakai dropbox.  It should be a link like https://XXXXXXXX.signin.aws.amazon.com/console

 

AWS Toolkit for Eclipse

AWS Information / Documentation

To install, first launch Eclipse.  Ideally, when you installed Eclipse for your last project, you chose to install "Eclipse IDE for Java EE Developers".  That has all the necessary Eclipse packages. No? Not sure?

Install Eclipse support packages. Choose Help->Install New Software.   Choose Work with: Indigio (http://download.eclipse.org/releases/indigo) (assuming that you have the latest "Indigo" version of Eclipse installed).  Now check the installation box next to the following packages:

  • Database Development -> Data Tools Platform Enablement Extender SDK
  • Database Development -> Data Tools Platform Extender SDK
  • Web Development -> Eclipse Java EE Developer Tools
  • Web Development -> Eclipse Java Web Developer Tools
  • Web Development -> Eclipse Web Developer Tools

Select next/finish and accept the licenses. Allow the installation to complete, and restart Eclipse.

Install AWS Toolkit for Eclipse. Choose Help->Install New Software.  Click the "Add" button next to "Work with".  Enter AWS for the name and http://aws.amazon.com/eclipse/ for the location, and click OK.  Now check the installation box next to the following packages:

  • AWS Toolkit for Eclipse - use the upper-level checkbox to include all the tools in this category

Select next/finish and accept the licenses. Allow the installation to complete, and restart Eclipse.

 

AWS Command-Line Tools for EC2 API

For Linux/Mac/Windows (tools are written in Java)

Documentation:

Installation:

  • Download zip file containing EC2 tools
  • Create a directory to hold it: elastic-compute-cli
  • Unzip file into directory
  • Create an environment variable called JAVA_HOME that points to a directory that contains the bin directory, which contains the java executable
  • Create an environment variable called EC2_HOME that points to the directory where you just unzipped the files to
  • Create an environment variable called EC2_PRIVATE_KEY that points to the EC2 private keypair you created previously via the AWS Web Console->EC2 tab
  • Create an environment variable called EC2_CERT that points to the X.509 certificate file (which we have not yet needed for any of the other tools)
    • IAM users under a master account (i.e. you!) don't have access to the Amazon web app that generates this certificate, but you can create the file manually and then tell Amazon to use it.
    • Generate the file based on your EC2 cert: openssl req -new -x509 -nodes -sha1 -days 3650 -key /path/to/YOUR_EC2_KEYPAIR.pem -outform PEM > ~/path/to/amazon_ec2_cert_x509.pem (on Linux)
    • Email me the contents of this new x509.pem file, since I have to enter it into your user account. There is an automated way to do this without my involvement (that doesn't involve emailing around a security key), but it would require you to install yet another CLI tool set...

Examples to verify library is functional:

  • List regions: ./bin/ec2-describe-regions
  • List availability zones inside your selected region: ./bin/ec2-describe-availability-zones (you can change your region if needed)
  • List EC2 keypairs: ./bin/ec2-describe-keypairs
  • List EC2 instances: ./bin/ec2-describe-instances (assuming that any EC2 instances are currently running)
  • .. and over 200 other commands! (Start/terminate EC2 instances, etc...)

 

AWS Command-Line Tools for Elastic Beanstalk

For Linux/Mac/Windows (tools are written in Ruby)

Documentation:

Installation:

  • Download zip file containing elastic beanstalk tools
  • Create a directory to hold it: elastic-beanstalk-cli
  • Unzip file into directory
  • Create a text file in the same directory as the ruby client, i.e. elastic-beanstalk-cli. Into this file, place the following information, customized for your specific account. 
    AWSAccessKeyId=XXX
    AWSSecretKey=XXX
  • Create an environment variable called AWS_CREDENTIAL_FILE that points to your credentials file
  • Install required software
    • Ruby: sudo apt-get install ruby-full (on Linux) or download from RubyForge (on Windows)
    • Rubygems with JSON gem:  sudo apt-get install rubygems libjson-ruby (on Linux)

Examples:

  • List configured applications:  ./bin/elastic-beanstalk-describe-applications
  • List configured application versions:  ./bin/elastic-beanstalk-describe-application-versions
  • Many other options! (Create/delete applications, create/delete/terminate environments, restart app servers)
    • Essentially, everything you can do through the AWS Management Console Web App you can do through here

 

AWS Command-Line Tools for Elastic MapReduce

For Linux/Mac/Windows (tools are written in Ruby)

Documentation:

Installation:

  • Download zip file containing elastic mapreduce tools
  • Create a directory to hold it: elastic-mapreduce-cli
  • Unzip file into directory
  • Install required software
    • Ruby: sudo apt-get install ruby-full (on Linux) or download from RubyForge (on Windows)
  • Create a text file called credentials.json in the same directory as the ruby client, i.e. elastic-mapreduce-cli. Into this file, place the following information, customized for your specific account and directory setup.  Note that this example is pre-set to the us-east-1 region, since that is where the CommonCrawl data is based. And, this example assumes you have already created your EC2 private key and saved it as a .pem file.
    {
    "access_id": "[YOUR AWS secret key ID]",
    "private_key": "[YOUR AWS secret key]",
    "keypair": "[YOUR key pair name]",
    "key-pair-file": "[The path and name of YOUR PEM file]",
    "log_uri": "s3n://[YOUR bucket name in S3]/mylog-uri/",
    "region": "us-east-1"
    }

Examples:

  • List available commands:  ./elastic-mapreduce --help
  • List running job flows: ./elastic-mapreduce --list
  • For more examples, see CommonCrawl Tutorial