AWS NACL vs Security Group

We can use AWS Network ACL (NACL) and Security Group to manage the security of VPC.

According to this article [1],

Security groups — Act as a firewall for associated Amazon EC2 instances, controlling both inbound and outbound traffic at the instance level
Network access control lists (ACLs) — Act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level

In more granular level, the differences are [1]:

Security Group Network ACL
Operates at the instance level Operates at the subnet level
Supports allow rules only Supports allow rules and deny rules
Is stateful: Return traffic is automatically allowed, regardless of any rules Is stateless: Return traffic must be explicitly allowed by rules
We evaluate all rules before deciding whether to allow traffic We process rules in number order when deciding whether to allow traffic
Applies to an instance only if someone specifies the security group when launching the instance, or associates the security group with the instance later on Automatically applies to all instances in the subnets it’s associated with (therefore, you don’t have to rely on users to specify the security group)

It sounds very easy, right? But in real work, the configuration of NACL and Security Group must be carefully planned. Otherwise, we might have some complicated scenarios.

One lesson I learned is that configuration of ephemeral port range on DMZ subnets are important so that my private subnets can function correctly.

NACL Inbound and Outbound Rules in DMZ subnet:

Rule # Type Protocol Port Range Source Allow / Deny
100 HTTP (80) TCP (6) 80 0.0.0.0/0 ALLOW
200 HTTPS (443) TCP (6) 443 0.0.0.0/0 ALLOW
300 SSH (22) TCP (6) 22 0.0.0.0/0 ALLOW
400 All ICMP – IPv4 ICMP (1) ALL 0.0.0.0/0 ALLOW
500 Custom TCP Rule TCP (6) 1024-65535 0.0.0.0/0 ALLOW
* ALL Traffic ALL ALL ::/0 DENY
* ALL Traffic ALL ALL 0.0.0.0/0 DENY

 

References:

  1. https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Security.html
  2. https://forums.aws.amazon.com/thread.jspa?threadID=180376
  3. https://forums.aws.amazon.com/thread.jspa?threadID=206362
  4. https://stackoverflow.com/questions/24861581/centos-could-not-retrieve-the-mirrorlist
Advertisements

AWS SNS vs SQS

Amazon Simple Notification Service (SNS) is a flexible, fully managed pub/sub messaging and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients [1].

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications [1].

Amazon Simple Queue Service (SQS) and Amazon SNS are both messaging services within AWS, which provide different benefits for developers. Amazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminating the need to periodically check or “poll” for updates. Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model, and can be used to decouple sending and receiving components. Amazon SQS provides flexibility for distributed components of applications to send and receive messages without requiring each component to be concurrently available. [3]

A common pattern is to use SNS to publish messages to Amazon SQS queues to reliably send messages to one or many system components asynchronously. [2], [4]

  SNS SQS
Design Pattern or Data Structure Publisher/Subscribe Queue
Push or Pull Model Push Pull
Message Storage No storage. Subscribers must be online to receive in Queue
Consumer Processing All online subscribers receive the message notification Only one consumer can process one message

References:

  1. https://aws.amazon.com/
  2. https://aws.amazon.com/sns/faqs/
  3. https://stackoverflow.com/questions/13681213/what-is-the-difference-between-amazon-sns-and-amazon-sqs
  4. https://aws.amazon.com/blogs/aws/queues-and-notifications-now-best-friends/

 

 

Create a On-the-Go Cloud Study Note Library using AWS Polly (Text-to-Speech generator)

Tags

Followed the example of a Cloud Guru, added Jian’s flavor, and created this on-the-go cloud study note generator using Amazon Polly (Text-to-Speech generator).

Please feel free to add your own text! Enjoy!

The technologies used are all in AWS: S3, S3 exposed as a static web site, API gateway, DynamoDB, SNS, and Polly.

https://s3.us-east-2.amazonaws.com/helpmestudyawspollyjian/index.html

 

 

TFS 2017 CI/CD Email Notification

It’s surprising to find out that there are no email notifications for release pipeline in TFS 2017. The feature is available in TFS 2018.

Instead, you have to go to azure devops market place, and find a 3rd party extension [5], Send Email. This extension works well, and allow you to use system and/or user variables defined in the pipeline. So you can implement a ‘Send Email’  task like this template.

Subject: RE: Deployment of release $(Release.ReleaseName) $(agent.jobstatus) in $(Release.EnvironmentName)

Email Body:

Release Definition: $(Release.DefinitionName)
Release: $(Release.ReleaseName)
Environment: $(Release.EnvironmentName)
Created by: $(Release.RequestedFor)

 

References:

  1. https://stackoverflow.com/questions/48383604/cant-create-release-notifications-in-tfs-2017
  2. https://roadtoalm.com/2016/08/11/set-output-variable-in-a-powershell-vsts-build-task/
  3. https://docs.microsoft.com/en-us/azure/devops/notifications/howto-manage-team-notifications?view=vsts
  4. https://blogs.msdn.microsoft.com/devops/2017/09/04/managing-release-notifications/
  5. https://marketplace.visualstudio.com/items?itemName=rvo.SendEmailTask

How to Remove 2017 TFS Agent

You might wonder why should we remove a TFS agent? There are many reasons. It could be that we don’t want that agent X at server Y any more, and we want to reclaim space.

Navigate to your agent location, and issue:

config.cmd remove

After that, we also need to delete that agent X folder manually.

 

References:

https://docs.microsoft.com/en-us/vsts/pipelines/agents/_shared/v2/remove-and-reconfigure-unix?view=vsts

https://docs.microsoft.com/en-us/vsts/pipelines/agents/v1-windows?view=tfs-2015&viewFallbackFrom=vsts

https://docs.microsoft.com/en-us/vsts/pipelines/agents/v2-windows?view=vsts&viewFallbackFrom=tfs-2015

https://stackoverflow.com/questions/42957331/force-removal-of-tfs-2017-build-agent

 

Default Values of Specific Version of DLLs in Visual Studio

We know ‘Specific Version’ is a compile time property. In run time, the correct version is always required.

If ‘Specific Version’ is known when a reference is made, it is very simple. If it’s true, the system will check the version. Otherwise, the system skips.

What if ‘Specific Version’ is not present? The default value will be decided according to this following table:

Reference Default Specific Version
<Reference Include=”System” /> False [because it’s just a name]
<Reference Include=”System.Web.Http.WebHost, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL”>

</Reference>

True [because it’s a strong name]

 

References:

https://stackoverflow.com/questions/1063459/net-reference-specificversion-true-or-false/1063536

https://stackoverflow.com/questions/24022134/how-exactly-does-the-specific-version-property-of-an-assembly-reference-work-i

https://social.msdn.microsoft.com/Forums/en-US/3a344927-c24d-49dc-a025-47c7efc29ddd/specific-versionfalse-and-gac-what-version-will-assembly-use?forum=csharpide

 

How to Stop/Terminate an Immortal AWS Elastic Beanstalk Instance?

After done with an AWS Elastic Beanstalk instance, tried to stop and then terminate it to regain some spaces back. To my surprise, no matter how I tried, AWS always automatically relaunched new instance.

It baffles me for a few days, and did some research. Some articles pointed to auto-scale configuration, and I then tried the suggested approach, and failed.

Armed with the hint of auto-scale configuration, I browsed around, and find these two sections, Auto-Scaling Group and Launch Configuration. Deleted related settings in these two sections, and my AWS Elastic Beanstalk instance terminated.

References:

https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/stop.html

https://serverfault.com/questions/806174/how-to-stop-elastic-beanstalk-without-terminating

https://forums.aws.amazon.com/thread.jspa?threadID=59027

 

 

 

AWS – User, Group, Role, and Policy

It’s very important to understand these concepts.

User: the person who interacts with AWS. The user has password and access keys, etc.

Group: a collection of users. For example, groups such as HR, IT, etc

Role: an entity that defines a set of permissions for making AWS service requests [2]. It’s more like a functional category of permission sets, and these permissions are for accessing AWS resources. Role doesn’t have password and access keys.

Policy: a document about permissions. It can attach to user, group, and role.

 

References

1. https://docs.aws.amazon.com/IAM/latest/UserGuide/id.html

2. https://aws.amazon.com/iam/faqs/

3. https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html

 

 

AWS – IAM Policies and Speculation on Implementations

Tags

AWS IAM policy document is written in JSON format. So it’s very easy to read and understand. I list a few samples below. From these policy documents, what do you think? I have a few speculations in terms of implementations.

  1. There is a parsing going on
  2. There is a permission tree after parsing
  3. There is a complex strategy for checking and granting these permissions.

Here is the question. How do we implement this complex strategy for checking and granting permissions? We should not use brute-force. We know there is a tree with some attributes attached to it. So we should try to look into best practices and design patterns. What do you think?

 

Sample AWS IAM Policy Documents

AdministratorAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "*",
            "Resource": "*"
        }
    ]
}

 

AmazonS3FullAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "*"
        }
    ]
}

 

AmazonS3ReadOnlyAccess

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": "*"
    }
  ]
}

 

Twelve-Factor App

Twelve-factor app is a design guideline for Software as a Service (SaaS). For more details about what it’s and for, please refer to ‘References’ section.

I came up with these to help me to remember quickly.

CDC  (Codebase, Dependencies, Config) – Central Disease Control

BBPP (Backing Services, Build/Release/Run, Processes, Port Binding) – British Petroleum x 2

CDP (Concurrency, Disposability, Dev/Prod Parity) – Career Development Path

LAp (Logs, Admin Processes)

 

1 A (Admin Processes)

2 B (Backing Services, Build/Release/Run)

3 C (Codebase, Config, Concurrency)

2 D (Dependencies, Disposability)

1 L (Logs)

3 P (Processes, Port Binding, Parity)

 

 

References:

https://12factor.net/

https://en.wikipedia.org/wiki/Twelve-Factor_App_methodology