Learning to be a code alchemist, one experiment at a time.

Python: What is a Decorator?

|

I was working on some python stuff when I stumbled upon the term decorator; not knowing what it was, a quick google search led to a relatively confusing explanation of what I believe to be a simple idea.

A decorator is simply a function that extends another function. As in, it executes that original function, but also does more. Take the function below:

def print_my_name: print(“Neil Patel”)

Now take this decorator below:

def my_decorator(some_function): def wrapper(): print(“Something is happening before some_function() is called.”) some_function() print(“Something is happening after some_function() is called.”) return(wrapper)

If we were to execute this line of code:

my_decorator(print_my_name)

This is what would happen:

Something is happening before some_function() is called. Neil Patel Something is happening after some_function() is called.


You can call a decorator on a function using the @ sign! Take a look at the timing function below:

import time def timing_function(some_function): def wrapper(): t1 = time.time() some_function() t2 = time.time() return “Time it took to run the function: “ + str((t2 - t1)) + “\n” return(wrapper)

@timing_function def my_function(): num_list = [] for num in (range(0, 10000)): num_list.append(num) print(“\nSum of all the numbers: “ + str((sum(num_list))))

print(my_function())

basically, what the @timing_function line does is say : my_function=timing_function(my_function)!

AWS Compute Week, Day 3: Serverless Day

|

Building Serverless Web Applications ?Randall Hunt

Technical Evangelist at AWS, of SpaceX and NASA randhunt@amazon.com @jrhunt on twitter Concepts, Design Patterns, Tooling, Demo AWS Lambda No servers to provision or manage Scales with Usage Never pay for idle Availability and fault tolerance built in 60TB/s throughput between S3 and Lambda Never pay for Idle (GB/s) Event driven, continuous Scaling Function versioning and aliases Versions = immutable copies of code+configurations Aliases=mutable pointers to versons Lambda Environment Variables Key-Value pairs Available via standard env variables, such as process.env or os.environ Can be encrypted via AWS KMS Web Applications, Backends, Data Processing, Chatbots, Amazon Alexa, Amazon API Gateway Concepts API gateway Variables Sam : Serverless Application Model (helps define entire Serverless application) is lot easier then cloud formation template Framework is zappa for python is best

Serverless Apps with AWS Step Functions

Trevor Roberts Jr, Solutions architect Modern Serverless Apps -> Lambda functions Take discrete units of work and doing them well and packing them into lambda functions 12 factor application AWS step functions: Application Lifecycle Json defined, Visualize in console, Monitor Execution Seven State types: Task, Choice, Parallel, Wait, Fail, Succeed, Pass

AWS Compute Week, Day 2: Docker

|

Docker With ECS (Elastic Container Service)

Why containers? OS virtualization, Process Isolation, Images, Automation Portable Flexible Fast Efficient Hard to switch over to batch jobs unless you have a cluster manager Amazon ECS is a cluster management tool Mix and match the different EC2 instances for your different container images ECS scheduling will do everything for you Optimistic, and strict scheduling Optimistic tries to reserve, if fails, then fail Other waits for completion before it works // block it out No built in queue in ECS Fully managed elastic service Shared state optimistic scheduling Deep integration with other AWS services Load balancing, block storage, etc. etc. Routing via Application Load balancer Path based routing Dynamic port mapping HTTP/2 WebSockets Detailed Logging Task Definition

Deep Dive on microservices and Docker/Containers

Microservices Architecture, ECS, Placement, and 12factor App What is a microservice? Small independent stateless process communicating with API?s. Small and focused on doing a small task which lets you have a modular approach to system building Monolithic vs Microservices Build it all inside a package and put it on a webserver (but makes it hard for new features or bugs) Make a change in the singular service you need Black boxed

1Codebase 2Dependencies 3Config 4Backing Services 5Build,Release Run 6Processes 7 Port Binding 8 Concurrencies 9 Disposability 10 Dev/Prod parity 11 Logs 12Admin Processes

Container Orchestration with Amazon ECS & Blox

New Cluster Query Language
Lot of different things to filter by Deep Dives on CI/CD and Docker What is CI/CD,using Docker and ECS/ECR for this Deployment strategies Frequncies reduce difficultiy Consistency improves confidence Automation over toil Empowered developers make happier teams Smaller batch sizes are easier to debug Faster delivery improves software development practices Version control, branching, code review, Compilation, unit tests, static analysis, packaging Integration tests, load tests, security tests, acceptance test Deployment, monitoring, measuring, validation

AWS Compute Week, Day 2: Deep Learning Workshop Introduction

|

Deploy a Deep Learning Framework on amazon ECS and EC2 Spot Instances

Introduce MXnet Containers Overview of ECS and ECR Overview of AWS CloudFormation Overview of EC2 Spot Instances ECS and EC2 Spot Instances Together Hands-on, Self paced workshop

What is MX net ?open source deep learning framework Define train and deploy deep neural networks. Highly scalable, multihost, cpu/gpu

Why containers? Increase infrastructure utilization, environment isolation and fidelity, run diverse applications on shared hardware, changes are tracked, easy to deploy. Increase portability, and flexibility, speed and efficiency.

ECR is EC2 container registry that you use to store your docker images It becomes a task definition and then reference an ECR container in the task

AWS Cloudformation Great for infrastructure as Code. USE JSON or Yaml files as templates, make a service and build the stack Stack replication and infrastructure scale out

Workshop: Image classification Lab 1: Setup the workshop environment on AWS Lab 2: Build an MXNet Docker Image Lab 3: Launch MXnet with ECS Lab 4: Image classification Demo Lab 5: Wrap Image Classification in an ECS Task

AWS Compute Week, Day 1: Spot, Batch, and Lightsail

|

Introduction to Spot

Learn about Spot and best practices customer examples Tools and partners to help manage spot instances See a demo Bid for unused EC2 capacity lying around in data centers AWS has millions of customers; 2300 government agencies, 7k schools, 22k nonprofit Spot market is a real time market where the price changes based on supply and demand Never pay more then what you bid; if price goes higher then that, you get 2 minutes Applications should be as stateless as possible Pick a bunch of types and zones and etc. etc. for more uptime 21 percent run < 1 hour 35 percent < 2 hours 40 percent < 3 hours 50 percent < less then 6 hours spot blocks : only get a spot instance if you can have for dedicated number of hours spot fleet: get lots of different kinds of spot instances EC2 Spot Console EC2 Spot Bid advisor EC2 Spot Labs on Github : Python scripts that do cool things How is market price set? Amazon sets it based on availability and bidding

AWS batch

Batch computing, overview, use cases, Demo, Q&A Execute a bunch of programs on a bunch of inputs 2007 NYT processed 130 years in 36 hours for 900 dollars in 2007 ; 11 million articles and 4TB Scale it Job: Unit of work executed by AWS batch as containerized application running on Amazon EC2; reference container, command parameters, or .zip(run the .zip on amazon linux container) Array jobs: Coming soon; run many copies of an application against an array of elements Job definitions: how to run the jobs; mount points etc. etc. Job queue: where job lives until assigned to compute environment Compute environment: the EC2 instances these jobs are going to run on managed(they do) unmanaged (you take care of the EC2) Job states: submitted (in queue), pending( waiting on dependencies), runnable(Ready to run), starting (your job is in process of being scheduled), succeeded, failed,

Introduction to AWS Lightsail

Everything you need for a low predictable price Mem, processor, SSD, Transfer (prorated as well) Launch a VPS with a single click Pick your own preconfigured images with the stacks installed and applications