TAGS

Software Architecture Infomation Security Programming Code Design Design Patterns XSD Semantic Web XML RuleML XSLT Aha! Prolog Software Design Maven MongoDB Python Data Mining NoSQL Node.js Java Spring Presentation Microservices Algorithms Maths Internet Express AWS Cloud Computing Type Script Ionic Javascript Testing HTML CSS SCSS JavaScript Architecture Cloud

AWS Cloud Computing Notes

Posted by monikma, 30 May 2020.
Programming Node.js Express AWS Microservices Cloud Computing Type Script Ionic Javascript Testing
Course notes

Those are the notes I took during the Cloud Developer Udacity Nanodegree Course, as well as during doing my projects in that topic.

Table of contents

Intro

Why cloud? Because this way we can handle more load and/or have more computational power.

A cluster is a collection of instances which perform the same function. An autoscaling group is a type of cluster that can increase or decrease the number of instances based on demand.

Tightly coupled systems are fast to stand-up but have a lot of technical debt. It is better to keep things loosely coupled. One extra advantage is that it is easy to outsource parts of the system. You should however not obsess too much over reducing technical debt, just keep in mind.

For REST API Design keep the resource endpoints to be nouns in their plural form.

Microservice Data Patterns

Greatly simplified:

Shared Data Pattern

Sometimes some microservices may use the same database, because it is too complicated to divide it into multiple databases.

Proxy Pattern

Forwards the requests from the frontend to the right microservice, based on the URL. Decouples the information of which service does what and how from the frontend, and even from the loadbalancer. E.g. nginx.

Aggregator Microservice Pattern

Like the Proxy Pattern above, but this service in the front is also a microservice with its own business logic. Can e.g. combine data from 2 services.

Chained Microservice Pattern

Service A needs data from Service B, which in turn needs data from Service C.

Branch Microservice Pattern

Service A needs data from Service B and from Service C.

Asynchronous Messaging Pattern

Async queue or publish/subscribe.

Environment setup

We will use Node Express and TypeScript. Course repo: https://github.com/udacity/cloud-developer.

How to install dependencies in Node:

Starting Node Express server

First, run this in the project folder:

npm install

Notice the following in the code, which will parse the JSON body automatically:

app.use(bodyParser.json());

To start the server run:

npm run dev

Testing

Third party libraries

Data Storage

NoSQL databases are easily scaled out, be careful not to create a mess with the dynamic schema just because you do not actually know what you are building. Relational databases are still a good choice in many cases, they are easy to scale up

Bloom Filters - interesting

Databases in AWS

Create SQL database

Allow access from outside (dev)

VPC (Virtual Private Cloud) is like a Firewall which will still block the connections from out dev machine. In order to fix that:

(you will get the two rows just by choosing “Anywhere”)

Filestores in AWS

They are good for storing binary data, cheaper than storing them in a database. AWS provides:

For uploading to AWS we will use the Signed URL Pattern: the client will get an expiring self-authenticating URL to upload the file themselves.

Create the S3 filestore

Update CORS for the S3 filestore

For Signed URL pattern to work, you need to update CORS for that Bucket.

* POST GET PUT DELETE HEAD * 

**Configure user and service access to the S3 bucket**

It is very important that any access in the cloud is revokable, and as limited as possible. Also, it is better have user groups rather than single policies added to each user (maintenance).

**IAM** - is Amazon's permission management. It applies to users and services.

To configure user access:

- create a new user (for dev env)
- check programmatic access (cli), AWS Console
- create user group and add the user to the group
- create new policy, to access the s3 bucket; we can create access only for this particular bucket (using ARN)
- download user's credentials (*)

To use the configured user access you need to install the IAM credentials locally:

- run `aws configure`
- enter the credentials you downloaded here (*); note this will be saved in ~/.aws/config (unix), you can see there that the default profile is called `default`

To configure service access:

- add a new server role for EC2 (that you will add in the future)
- add the new policy to the role


## Deploy your service to AWS

This is about putting what we have locally into the cloud. AWS offers a deployment tool called **Elastic Beanstalk**. It adds Load Balancer, NGIX Proxy (ssl, queuing requests if too many), automatic scaling.

*** Temporarily, you need to add an admin policy to your dev user, to make him be able to crete the EB. Later you can remove it.

We can use the distribution for Node.

- download and install [EB Cli](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install.html) (for Windows it may sound complicated, but don't give up, remember you can run the cmd in Administrator mode in Windows too)
- run `eb init`
- choose no codeCommit
- choose to setup SSH (for generating SSH pair, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)

The configuration will be written into `config.yml`. Add there:

deploy: artifact: ./www/Archive.zip

- for Windows install [UnxUtils](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/nodejs-dynamodb-tutorial.html#nodejs-dynamodb-tutorial-prereqs) (stay brave)

In the `package.json` -> script -> build there is a little trick of preparing that `*.zip` file that AWS accepts, all in one line.

(`.npmrc` is a file for npm, tells it how to install dependencies)

- run `npm run build` to have the `Archive.zip` created
- run `eb create` to have everything deployed to AWS with Elastic Beanstalk
- you can take the defaults, for Load Balancer choose "Application"
- in AWS Console update the env variables (Configuration -> Software -> Modify -> Env. variables)

Note that the profile is now `DEPLOYED`, this is because in the code there is an `if` not to copy the IAM local credentials in such case:

if(c.aws_profile !== “DEPLOYED”) { var credentials = new AWS.SharedIniFileCredentials({profile: ‘default’}); AWS.config.credentials = credentials; }

- finally, add our new s3 bucket policy to the role of that EC2 (lazy solution, normally you'd create a dedicated role)

*** You can now remove the admin policy from your dev user.

#### Deploying changes

If you need to deploy code changes to AWS:
- `npm run build`
- `eb deploy`

## Security

Common errors:
- do not log passwords
- do not commit passwords to git
- do not send passwords without SSL
- do not store passwords plaintext, hash them
- to avoid rainbow table attack in case of hash leak, hash them with a random (long enough) salt, using `bcrypt` not `MD5` (as of 2019)

Salt is generated also using hash algorithm/library. Salt is stored in the database, together with the hashed password. 

**Hash and compare the user password with bcrypt**

To hash it:

const rounds = 10; //2^10 rounds const salt = await bcrypt.genSalt(rounds); const hash = await bcrypt.hash(plain, salt);


To compare the password with the hash (under the hood the salt is extracted from the hash to hash the `plain` with it):

await bcrypt.compare(plain, hash);


### Storing user authentication in the browser

Once the user enters their correct password, the server can respond with an information that the browser can use for a limited time, in order to authenticate the user without them needing to re-enter the password each time. For example:
- server responds with a session id that is stored in a local cookie
- server responds with a [JWT token](https://jwt.io/introduction/) that is stored e.g. in Browser's local storage; *In short JWT token is a self signed content that the author (in that case the server) can then decrypt later;*

### Storing service authentication in the service

That happens using authorization tokens. The server gets a token from another server, and can use it for subsequent calls. Remember, store the secrets in environment variable.

### JWT tokens

(A JWT contains `header`.`user data`.`the two encoded`, encoded with a secret that is known only by the server. So that is how it is read-only. And if 2 services use same secret then we have SSO out of the box.)

#### Creating JWT

return jwt.sign(user, config.jwt.secret);


#### Verifying JWT

We add a `requireAuth` function as an [Express Middleware](https://expressjs.com/en/guide/writing-middleware.html).

export function requireAuth(req: Request, res: Response, next: NextFunction) { if (!req.headers || !req.headers.authorization){ return res.status(401).send({ message: ‘No authorization headers.’ }); }

const token_bearer = req.headers.authorization.split(' ');
if(token_bearer.length != 2){
    return res.status(401).send({ message: 'Malformed token.' });
}

const token = token_bearer[1];

return jwt.verify(token, "hello", (err, decoded) => {
  if (err) {
    return res.status(500).send({ auth: false, message: 'Failed to authenticate.' });
  }
  return next();
}); } ```

In the router we protect the endpoint with requireAuth like this:

router.patch('/:id',
    requireAuth, 
    async (req: Request, res: Response) => {
    ...

Managing user credentials in AWS

AWS Cognito.

Security resources

You should have a Security Policy at your company.

Security Policy examples:

Other resources:

Maintenance, scaling, etc

DNS

Amazon’s DNS is called Route 53. Functionalities:

It allows you to point directly to the AWS infrastructure (e.g. referring by resource name).

DNS stores:

Ionic client with Angular - supre basics

With Ionic (also React) you can move your app to the mobile using something called hybrid app.

You need Ionic cli installed globally (see the link).

CDN

You can put the whole frontend into the CDN.

For that you need to:

Scaling

AWS can automatically manage instances depending on the load - that is called auto-scaling.

Scaling up

In AWS there are instance types - specialized to better handle certain situations. This you set in Elastic Beanstalk.

Scaling out

Is adding more server instances, after scaling up is not possible anymore. Also Load Balancer comes into play then.

We also configure it in Elastic Beanstalk:

See Scaling triggers - there you can set the thresholds. The documentation is here.

Database

Can also be scaled, up and out. See the docu.

Debugging on the cloud

Troubleshooting:

If Elastic Beanstalk status is red

Other tools:

Comments


Comments: