Those are the notes I took during the Cloud Developer Udacity Nanodegree Course, as well as during doing my projects in that topic.
Why cloud? Because this way we can handle more load and/or have more computational power.
A cluster is a collection of instances which perform the same function. An autoscaling group is a type of cluster that can increase or decrease the number of instances based on demand.
Tightly coupled systems are fast to stand-up but have a lot of technical debt. It is better to keep things loosely coupled. One extra advantage is that it is easy to outsource parts of the system. You should however not obsess too much over reducing technical debt, just keep in mind.
For REST API Design keep the resource endpoints to be nouns in their plural form.
Greatly simplified:
Shared Data Pattern
Sometimes some microservices may use the same database, because it is too complicated to divide it into multiple databases.
Proxy Pattern
Forwards the requests from the frontend to the right microservice, based on the URL. Decouples the information of which service does what and how from the frontend, and even from the loadbalancer. E.g. nginx.
Aggregator Microservice Pattern
Like the Proxy Pattern above, but this service in the front is also a microservice with its own business logic. Can e.g. combine data from 2 services.
Chained Microservice Pattern
Service A needs data from Service B, which in turn needs data from Service C.
Branch Microservice Pattern
Service A needs data from Service B and from Service C.
Asynchronous Messaging Pattern
Async queue or publish/subscribe.
We will use Node Express and TypeScript. Course repo: https://github.com/udacity/cloud-developer.
How to install dependencies in Node:
npm i bcrypt --save - install bcrypt, and add it to package.json dependenciesnpm i --save-dev @types/bcrypt - install types for bcrypt and add it to package.json developer dependenciesFirst, run this in the project folder:
npm install
Notice the following in the code, which will parse the JSON body automatically:
app.use(bodyParser.json());
To start the server run:
npm run dev
Testing
Third party libraries
NoSQL databases are easily scaled out, be careful not to create a mess with the dynamic schema just because you do not actually know what you are building. Relational databases are still a good choice in many cases, they are easy to scale up
Bloom Filters - interesting
Create SQL database
Allow access from outside (dev)
VPC (Virtual Private Cloud) is like a Firewall which will still block the connections from out dev machine. In order to fix that:
Postgres TCP 5432 0.0.0.0
Postgres TCP 5432 ://0
(you will get the two rows just by choosing “Anywhere”)
They are good for storing binary data, cheaper than storing them in a database. AWS provides:
For uploading to AWS we will use the Signed URL Pattern: the client will get an expiring self-authenticating URL to upload the file themselves.
Create the S3 filestore
Update CORS for the S3 filestore
For Signed URL pattern to work, you need to update CORS for that Bucket.
**Configure user and service access to the S3 bucket**
It is very important that any access in the cloud is revokable, and as limited as possible. Also, it is better have user groups rather than single policies added to each user (maintenance).
**IAM** - is Amazon's permission management. It applies to users and services.
To configure user access:
- create a new user (for dev env)
- check programmatic access (cli), AWS Console
- create user group and add the user to the group
- create new policy, to access the s3 bucket; we can create access only for this particular bucket (using ARN)
- download user's credentials (*)
To use the configured user access you need to install the IAM credentials locally:
- run `aws configure`
- enter the credentials you downloaded here (*); note this will be saved in ~/.aws/config (unix), you can see there that the default profile is called `default`
To configure service access:
- add a new server role for EC2 (that you will add in the future)
- add the new policy to the role
## Deploy your service to AWS
This is about putting what we have locally into the cloud. AWS offers a deployment tool called **Elastic Beanstalk**. It adds Load Balancer, NGIX Proxy (ssl, queuing requests if too many), automatic scaling.
*** Temporarily, you need to add an admin policy to your dev user, to make him be able to crete the EB. Later you can remove it.
We can use the distribution for Node.
- download and install [EB Cli](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3-install.html) (for Windows it may sound complicated, but don't give up, remember you can run the cmd in Administrator mode in Windows too)
- run `eb init`
- choose no codeCommit
- choose to setup SSH (for generating SSH pair, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)
The configuration will be written into `config.yml`. Add there:
deploy: artifact: ./www/Archive.zip
- for Windows install [UnxUtils](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/nodejs-dynamodb-tutorial.html#nodejs-dynamodb-tutorial-prereqs) (stay brave)
In the `package.json` -> script -> build there is a little trick of preparing that `*.zip` file that AWS accepts, all in one line.
(`.npmrc` is a file for npm, tells it how to install dependencies)
- run `npm run build` to have the `Archive.zip` created
- run `eb create` to have everything deployed to AWS with Elastic Beanstalk
- you can take the defaults, for Load Balancer choose "Application"
- in AWS Console update the env variables (Configuration -> Software -> Modify -> Env. variables)
Note that the profile is now `DEPLOYED`, this is because in the code there is an `if` not to copy the IAM local credentials in such case:
if(c.aws_profile !== “DEPLOYED”) { var credentials = new AWS.SharedIniFileCredentials({profile: ‘default’}); AWS.config.credentials = credentials; }
- finally, add our new s3 bucket policy to the role of that EC2 (lazy solution, normally you'd create a dedicated role)
*** You can now remove the admin policy from your dev user.
#### Deploying changes
If you need to deploy code changes to AWS:
- `npm run build`
- `eb deploy`
## Security
Common errors:
- do not log passwords
- do not commit passwords to git
- do not send passwords without SSL
- do not store passwords plaintext, hash them
- to avoid rainbow table attack in case of hash leak, hash them with a random (long enough) salt, using `bcrypt` not `MD5` (as of 2019)
Salt is generated also using hash algorithm/library. Salt is stored in the database, together with the hashed password.
**Hash and compare the user password with bcrypt**
To hash it:
const rounds = 10; //2^10 rounds const salt = await bcrypt.genSalt(rounds); const hash = await bcrypt.hash(plain, salt);
To compare the password with the hash (under the hood the salt is extracted from the hash to hash the `plain` with it):
await bcrypt.compare(plain, hash);
### Storing user authentication in the browser
Once the user enters their correct password, the server can respond with an information that the browser can use for a limited time, in order to authenticate the user without them needing to re-enter the password each time. For example:
- server responds with a session id that is stored in a local cookie
- server responds with a [JWT token](https://jwt.io/introduction/) that is stored e.g. in Browser's local storage; *In short JWT token is a self signed content that the author (in that case the server) can then decrypt later;*
### Storing service authentication in the service
That happens using authorization tokens. The server gets a token from another server, and can use it for subsequent calls. Remember, store the secrets in environment variable.
### JWT tokens
(A JWT contains `header`.`user data`.`the two encoded`, encoded with a secret that is known only by the server. So that is how it is read-only. And if 2 services use same secret then we have SSO out of the box.)
#### Creating JWT
return jwt.sign(user, config.jwt.secret);
#### Verifying JWT
We add a `requireAuth` function as an [Express Middleware](https://expressjs.com/en/guide/writing-middleware.html).
export function requireAuth(req: Request, res: Response, next: NextFunction) { if (!req.headers || !req.headers.authorization){ return res.status(401).send({ message: ‘No authorization headers.’ }); }
const token_bearer = req.headers.authorization.split(' ');
if(token_bearer.length != 2){
return res.status(401).send({ message: 'Malformed token.' });
}
const token = token_bearer[1];
return jwt.verify(token, "hello", (err, decoded) => {
if (err) {
return res.status(500).send({ auth: false, message: 'Failed to authenticate.' });
}
return next();
}); } ```
In the router we protect the endpoint with requireAuth like this:
router.patch('/:id',
requireAuth,
async (req: Request, res: Response) => {
...
AWS Cognito.
You should have a Security Policy at your company.
Security Policy examples:
Other resources:
Amazon’s DNS is called Route 53. Functionalities:
It allows you to point directly to the AWS infrastructure (e.g. referring by resource name).
DNS stores:
With Ionic (also React) you can move your app to the mobile using something called hybrid app.
You need Ionic cli installed globally (see the link).
ionic serve to start the frontend server.http://localhost:8100/homeYou can put the whole frontend into the CDN.
For that you need to:
ionic build - will process TS files to JS, also process HTMLionic build --prod - with use settings saved in environment.prod.tsAWS can automatically manage instances depending on the load - that is called auto-scaling.
In AWS there are instance types - specialized to better handle certain situations. This you set in Elastic Beanstalk.
Is adding more server instances, after scaling up is not possible anymore. Also Load Balancer comes into play then.
We also configure it in Elastic Beanstalk:
See Scaling triggers - there you can set the thresholds. The documentation is here.
Can also be scaled, up and out. See the docu.
Troubleshooting:
If Elastic Beanstalk status is red
Other tools:
Comments: