Cloud for Data Scientists
TL,DR: cloud knowledge cannot be overestimated as a super power for data scientist. Here I discuss six key cloud tools that literally changed my professional life.
Why would data scientist need the cloud? Last week my new MacBook was broken (long story short: the machine politely asked me update the operating system , I accepted and then realized one should never upgrade OS on the train using smartphone connectivity.) The damage: cannot login, need to erase everything. To keep my professional name I won’t disclose the thing that I have lost, but let’s just say that sometimes even when my code is properly deployed in git — some of things I value are, well, dataset. Properly using the cloud could significantly reduce our dependancy on the laptop. Furthermore, use of cloud resources can unlock some superpowers. Here I’ll discuss the basics of setting up AWS cloud account, including AWS S3 (storage), computing instance (a virtual computer), ssh tunnels to work locally on a remote Jupyter notebook, some relevant unix commands including rsync to upload code safely and easily, crontab and finally volumes to backup instance state.
Getting started: open AWS account
Go to the Amazon Web Services home page. Choose Create an AWS Account. Note: If you’ve signed in to AWS recently, the button might say Sign In to the Console. Enter your account information, and then choose Continue (later on they’ll send a confirmation emails). Choose professional account, enter your company information. If Create a new AWS account isn’t visible, first choose Sign in to a different account, and then choose Create a new AWS account, accept the agreement, Choose Create Account and Continue. Activate your account and add a payment method on the Payment Information page, choose Secure Submit, and verify using phone SMS. Choose an AWS Support plan: for now choose the Basic Support. This plan is included for all AWS customers and include: Customer Service & Communities — 24x7 access to customer service, documentation, white papers, and support forums. AWS Trusted Advisor — Access to the 7 core Trusted Advisor checks and guidance to provision your resources following best practices to increase performance and improve security. AWS Personal Health Dashboard — A personalized view of the health of AWS services, and alerts when your resources are impacted. Even though this plan is free, AWS customer support engineers are responsive and knowledgeable. Don’t hesitate to ask anything. For example, if you don’t understand the billing, if you paid for something you thought was free — very often you’d get a refund. Higher support options are also available, for things that you typically pay for (for example — instruction on how to setup SSL on your server). Wait for account activation. After you choose a Support plan, a confirmation page indicates that your account is being activated. Accounts are usually activated within a few minutes, but the process might take up to 24 hours. You can sign in to your AWS account during this time. The AWS home page might display a button that shows “Complete Sign Up” during this time, even if you’ve completed all the steps in the sign-up process. When your account is fully activated, you’ll receive a confirmation email. After you receive this email, you have full access to all AWS services.
We will contact you as soon as possible.