DevOps Tip – Use PowerShell to Manage IIS

Tags

Just for fun. We spend some time on navigating IIS using Powershell. We can browse IIS using PowerShell ISE just like any file system. You can use commands such as cd, dir, etc.

Navigation

Import-module -Name WebAdministration

cd IIS:\

dir

cd Sites

cd “Default Web Site”

Create App Pool

New-WebAppPool -Name MyWebSite | Set-ItemProperty -name processModel.identityType -Value “AppPoolIdentity”

Modify Default Web Site

Set-ItemProperty IIS:\Sites\”Default Web Site” -name physicalPath -value “X:\MyDir\MyWebsite”
Set-ItemProperty IIS:\Sites\”Default Web Site” -Name applicationPool -Value “MyWebSite”

Create Virtual Directory and Web App

New-WebVirtualDirectory -Name “MyDir\MySubDir” -PhysicalPath MyLocation -Site “Default Web Site”
New-WebApplication -Name MyApp -ApplicationPool MyPool -PhysicalPath MyLocation -Site “Default Web Site/MyApp”

Authentication Configuration

Get-WebConfiguration system.webServer/security/authentication/* /* -Recurse | where {$_.enabled -eq $true} | format-list

Set-WebConfiguration system.webServer/security/authentication/anonymousAuthentication -PSPath MACHINE/WEBROOT/APPHOST -Location “Default Web Site/MyApp” -Value @{enabled=”False”}
Set-WebConfiguration system.webServer/security/authentication/windowsAuthentication -PSPath MACHINE/WEBROOT/APPHOST -Location “Default Web Site/MyApp” -Value @{enabled=”False”}

File Shares

New-SmbShare -Name MyWebsite -Path X:\Web\MyWebsite -Description MyWebsite -FullAccess “domain users”, “administrators” -ReadAccess “users”

Advertisements

DevOps Tip – Decouple Database Changes from Application Changes

Tags

One of DevOps ideas is “Decouple database changes from application changes”.

The basic ideas from DBAs are:

  1. DBAs only make additive changes to objects in databases
  2. DBAs make no assumptions about what version of objects will be used in different environments

At the same time, front-end and business layer developers should use the feature toggle concept. The increments should support multiple database object versions.

Launch a Command Line from the Main Program

Sometimes, we need to launch a command line to execute the output from the main program. 

public static class CmdExecuteUtil
{
  private static void ExecuteCommand(string command)
  {
    Process cmd = new Process();

    cmd.StartInfo.FileName = "cmd.exe";
    cmd.StartInfo.RedirectStandardInput = true;
    cmd.StartInfo.RedirectStandardOutput = true;
    cmd.StartInfo.CreateNoWindow = true;
    cmd.StartInfo.UseShellExecute = false;

    cmd.Start();

    cmd.StandardInput.WriteLine(command);
    cmd.StandardInput.Flush();
    cmd.StandardInput.Close();
    Console.WriteLine(cmd.StandardOutput.ReadToEnd());
  }

  public static void ExecuteCheckoutCommand(string command)
  {
    ExecuteCommand("sdk command or cli, etc" + command);
  }
}


Udemy 503, Service Unavailable – A Black Friday Looks Like

Here is what the Black Friday looks like on Udemy when you click to study a course, a 503 status code, Service Unavailable. I would guess Udemy is hosted on cloud already. However, even the cloud cannot scale it up or out anymore during Black Friday. under these extreme conditions, it is running in a Pilot Light mode, only minimal critical services such as searching and purchasing are kept.

Udemy-503

Remotely Execute PowerShell Database Scripts in CI/CD Pipeline

There are many ways of deploying database scripts. Due to the constraints, you might have to choose not the best way, deploy the scripts to target machine, and then execute them remotely from CI/CD pipeline.

# References [2, 3, 4]
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100

# References [1]
#Provide SQLServerName
$SQLServer =”servername”
#Provide Database Name
$DatabaseName =”dbname”
#Scripts Folder Path
$FolderPath =”folderpath”

#Loop through the .sql files and run them
foreach ($filename in get-childitem -path $FolderPath -filter “*.sql”)
{
invoke-sqlcmd –ServerInstance $SQLServer -Database $DatabaseName -InputFile              $filename.fullname
#Print file name which is executed
$filename.fullname
}

References:

  1. http://www.techbrothersit.com/2015/08/how-to-execute-sql-files-from-directory.html
  2. https://social.msdn.microsoft.com/Forums/en-US/cb85aae2-0c3c-4077-a525-37e93e08b506/getting-sqlservercmdletsnapin100-inside-powershell?forum=sqlsmoanddmo
  3. https://stackoverflow.com/questions/42762989/invoke-sqlcmd-unable-to-run
  4. https://blog.sqlauthority.com/2014/11/23/sql-server-fix-the-term-invoke-sqlcmd-is-not-recognized-as-the-name-of-a-cmdlet/

 

Other notes:

Set-PSSessionConfiguration -ShowSecurityDescriptorUI -Name Microsoft.PowerShell

get-module -listavailable

Get-PSSnapin -Registered

Microsoft.SqlServer.Management.PSSnapin

Is Daily Deployment a Dream for Traditional IT Shops?

Business divisions always pushes for their agenda to be released as quickly as possible. Most development teams in traditional IT shops release every two, three, or four weeks. Every release is very stressful for development teams because it’s usually at odd hours, and development teams have to be present and support during release, and rollback if things go wrong.

I remembered long time ago at a client site. We set up a deployment window from 11:00am to 12:00am every night. The entry website was designed to have a gradual lockdown from showing the maintenance window warning messages, minutes count down, and final lockdown and redirecting users to a maintenance page with a link back to regular website when it’s back online.

After a while, the users were trained to get used to this maintenance window. We were able to deploy small tasks during the window. It worked very well because the tasks in daily deployment were so small.

When the tasks are small, the deployment is fast. If anything goes wrong, we can easily identify and correct the issues, and deploy the fixes the next day during maintenance window. In addition, we can implement feature toggles in those tasks, might simply turn off the features so that we don’t need to patch the system immediately.

AWS NACL vs Security Group

We can use AWS Network ACL (NACL) and Security Group to manage the security of VPC.

According to this article [1],

Security groups — Act as a firewall for associated Amazon EC2 instances, controlling both inbound and outbound traffic at the instance level
Network access control lists (ACLs) — Act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level

In more granular level, the differences are [1]:

Security Group Network ACL
Operates at the instance level, and port level Operates at the subnet level, and IP address level
Supports allow rules only Supports allow rules and deny rules
Is stateful: Return traffic is automatically allowed, regardless of any rules Is stateless: Return traffic must be explicitly allowed by rules
We evaluate all rules before deciding whether to allow traffic We process rules in number order when deciding whether to allow traffic
Applies to an instance only if someone specifies the security group when launching the instance, or associates the security group with the instance later on Automatically applies to all instances in the subnets it’s associated with (therefore, you don’t have to rely on users to specify the security group)

It sounds very easy, right? But in real work, the configuration of NACL and Security Group must be carefully planned. Otherwise, we might have some complicated scenarios.

One lesson I learned is that configuration of ephemeral port range on DMZ subnets are important so that my private subnets can function correctly.

NACL Inbound and Outbound Rules in DMZ subnet:

Rule # Type Protocol Port Range Source Allow / Deny
100 HTTP (80) TCP (6) 80 0.0.0.0/0 ALLOW
200 HTTPS (443) TCP (6) 443 0.0.0.0/0 ALLOW
300 SSH (22) TCP (6) 22 0.0.0.0/0 ALLOW
400 All ICMP – IPv4 ICMP (1) ALL 0.0.0.0/0 ALLOW
500 Custom TCP Rule TCP (6) 1024-65535 0.0.0.0/0 ALLOW
* ALL Traffic ALL ALL ::/0 DENY
* ALL Traffic ALL ALL 0.0.0.0/0 DENY

The client that initiates the request chooses the ephemeral port range. For example, if a request comes into a web server in your VPC from a browser on the Internet, your network ACL must have an outbound rule to enable traffic destined for ports 1024-65535. If an instance in your VPC is the client initiating a request, your network ACL must have an inbound rule to enable traffic destined for the ephemeral ports specific to the type of instance [5]

References:

  1. https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Security.html
  2. https://forums.aws.amazon.com/thread.jspa?threadID=180376
  3. https://forums.aws.amazon.com/thread.jspa?threadID=206362
  4. https://stackoverflow.com/questions/24861581/centos-could-not-retrieve-the-mirrorlist
  5. https://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.htmlhttps://docs.aws.amazon.com/vpc/latest/userguide/vpc-network-acls.html

AWS SNS vs SQS

Amazon Simple Notification Service (SNS) is a flexible, fully managed pub/sub messaging and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients [1].

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications [1].

Amazon Simple Queue Service (SQS) and Amazon SNS are both messaging services within AWS, which provide different benefits for developers. Amazon SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminating the need to periodically check or “poll” for updates. Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model, and can be used to decouple sending and receiving components. Amazon SQS provides flexibility for distributed components of applications to send and receive messages without requiring each component to be concurrently available. [3]

A common pattern is to use SNS to publish messages to Amazon SQS queues to reliably send messages to one or many system components asynchronously. [2], [4]

  SNS SQS
Design Pattern or Data Structure Publisher/Subscribe Queue
Push or Pull Model Push Pull
Message Storage No storage. Subscribers must be online to receive in Queue
Consumer Processing All online subscribers receive the message notification Only one consumer can process one message

References:

  1. https://aws.amazon.com/
  2. https://aws.amazon.com/sns/faqs/
  3. https://stackoverflow.com/questions/13681213/what-is-the-difference-between-amazon-sns-and-amazon-sqs
  4. https://aws.amazon.com/blogs/aws/queues-and-notifications-now-best-friends/

 

 

Create a On-the-Go Cloud Study Note Library using AWS Polly (Text-to-Speech generator)

Tags

Followed the example of a Cloud Guru, added Jian’s flavor, and created this on-the-go cloud study note generator using Amazon Polly (Text-to-Speech generator).

Please feel free to add your own text! Enjoy!

The technologies used are all in AWS: S3, S3 exposed as a static web site, API gateway, DynamoDB, SNS, and Polly.

https://s3.us-east-2.amazonaws.com/helpmestudyawspollyjian/index.html

 

 

TFS 2017 CI/CD Email Notification

It’s surprising to find out that there are no email notifications for release pipeline in TFS 2017. The feature is available in TFS 2018.

Instead, you have to go to azure devops market place, and find a 3rd party extension [5], Send Email. This extension works well, and allow you to use system and/or user variables defined in the pipeline. So you can implement a ‘Send Email’  task like this template.

Subject: RE: Deployment of release $(Release.ReleaseName) $(agent.jobstatus) in $(Release.EnvironmentName)

Email Body:

Release Definition: $(Release.DefinitionName)
Release: $(Release.ReleaseName)
Environment: $(Release.EnvironmentName)
Created by: $(Release.RequestedFor)

 

References:

  1. https://stackoverflow.com/questions/48383604/cant-create-release-notifications-in-tfs-2017
  2. https://roadtoalm.com/2016/08/11/set-output-variable-in-a-powershell-vsts-build-task/
  3. https://docs.microsoft.com/en-us/azure/devops/notifications/howto-manage-team-notifications?view=vsts
  4. https://blogs.msdn.microsoft.com/devops/2017/09/04/managing-release-notifications/
  5. https://marketplace.visualstudio.com/items?itemName=rvo.SendEmailTask