Holy on Dev
An answer to CircleCI''s "Why we’re no longer using Core.typed"
CircleCI has recently published a very useful post "Why we’re no longer using Core.typed" that raises some important concerns w.r.t. Typed Clojure that in their particular case led to the cost overweighting the benefits. CircleCI has a long and positive relation to Ambrose Bonnaire-Sergeant, the main author of core.typed, that has addressed their concerns in his recent Strange Loop talk "Typed Clojure: From Optional to Gradual Typing" (gradual typing is also explained in his 6/2015 blog post "Gradual typing for Clojure"). For the sake of searchability and those of us who prefer text to video, I would like to summarise the main points from the response (spiced with some thoughts of my own).
Refactoring & Type Errors in Clojure: Experience and Prevention
While refactoring a relatively simple Clojure code to use a map instead of a vector, I have wasted perhaps a few hours due to essentially type errors. I want to share the experience and my thoughts about possible solutions since I encounter this problem quite often. I should mention that it is quite likely that it is more a problem (an opportunity? :-)) with me rather than the language, namely with the way I write and (not) test it.
The core of the problem is that I write chains of transformations based on my sometimes flawed idea of what data I have at each stage. The challenge is that I cannot see what the data is and have to maintain a mental model while writing the code, and I suck at it. Evaluating the code in the REPL as I develop it helps somewhat but only when writing it - not when I decide to refactor it.
Nginx: Protecting upstream from overload on cache miss
These 2 magical lines will protect your upstream server from possible overload of many users try to access the same in cached or expired content:
Shipping a Refactoring & Feature One Tiny Slice at a Time, to Reduce Risk
You don’t need to finish a feature and your users don’t need to see it to be able to release and start battle-testing it. Slice it as much as possible and release the chunks ASAP to shorten the feedback loop and decrease risk.
My colleagues have been working on a crucial change in our webshop - replacing our legacy shopping cart and checkout process with a new one and implementing some new, highly desired functionality that this change enables. We have decided to decrease the risk of the change by doing it first only for product accessories. However the business wanted the new feature included and that required changes to the UI. But the UI has to be consistent across all sections so we would need to implement it also for the main products before going live - which would necessitate implementing also the more complex process used by the main products (and not yet supported by the new backend). And suddenly we had a a load of work that would take weeks to complete and would be released in a big bang deployment.
Such a large-scale and time-consuming change without any feedback from reality whatsoever and then releasing it all at once, having impact on all our sales - I find that really scary (and have fought it before). It is essentially weeks of building risk and then releasing it in a big kaboom. How could we break it down, to release it in small slices, without making the business people unhappy?
AWS ebextensions: Avoiding "Could not enable service" (or .. disable ..)
If you are adding a service entry to your .ebextensions/ config to run a service in AWS Elastic Beanstalk and it fails with either "Could not enable service [..]" or "Could not disable service [..]" (based on the value of ensureRunning), make sure that the service init.d file supports chkconfig, i.e. contains the comments it looks for.
Running Gor, the HTTP traffic replayer, as a service on AWS Elastic Beanstalk
Gor is a great utility for replicating (a subset of) production traffic to a staging/test environment. Running it on AWS Elastic Beanstalk (EB) has some challenges, mainly that it doesn't support running as a daemon and that there isn't any documentation/examples for doing this. Well, here is a solution:
AWS: Passing private configuration to a Docker container (via S3)
Philipp Garbe describes how to pass environment variables that you want to keep private to a public Docker instance run on Amazon Web Services (beanstalk or ECS) in his post How to Run HuBot in Docker on AWS EC2 Container Services - Part 3. The trick is:
Fixing a mysterious .ebextensions command time out (AWS Elastic Beanstalk)
Our webshop, nettbutikk.netcom.no, runs on AWS Elastic Beanstalk and we use
.ebextensions/to customize the environment. I have been just trying to get Gor running on our leader production instance to replay some traffic to our staging environment so that we get a much richer feedback from it. However the
container_commandI used caused the instance to time out and trash the environment, against all reason. The documentation doesn't help and troubleshooting this is hard due to lack of feedback and time-consuming. Luckily I have arrived to a solution.
Book Review & Digest: Release It! Design and Deploy Production-Ready Software
By Michael T. Nygard, 2007, ISBN: 978-0-9787-3921-8
My digest and review of the book.
Of the books I have read, Release It! is the one I would require all "senior" developers to read (together with something like Architecting Enterprise Solutions: Patterns for High-Capability Internet-based Systems). Especially the first part on stability with its patterns and anti-patterns is a must read. Without knowing and applying them, we create systems that react to problems like a dry savannah to a burning match. I found also to next to last chapter, #17 Transparency, very valuable, especially the metrics and design of the OpsDB and observation practices.
One thing I have left out of the digest which is really worth reading are the war stories that introduce each section, they are really interesting, inspiring, and educational.
- Release It! slides: http://gotocon.com/dl/jaoo-sydney-2009/slides/MichaelT.Nygard_FailureComesInFlavoursPart2.pdf
- Netflix's Dependency Command talks about using circuit breakers and a thread pool limit http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
- MSDN Guidance for Cloud Applications: Design Patterns - Circuit Breaker and many more highly useful patterns
Stability x longevity bugs
Integration point = call to a DB, WS, ... . Stability risk #1.
NDC Oslo 2015: Talk notes, recommended talks (security, FP, etc.)
A great conference. A good deal of good talks.
To (perhaps) check later
- Practical CSS tips & tricks for backend developers - really useful tips! video
- No Estimates, Let's Explore the Possibilities video
- Form with Function: Adding Behavior with CSS - recommended by a friend (modal dialogs, tab switching, ...) video
- ? Anti-fragile and feedback. Trying to make up for the failures of "agile." video
- ? Designing and Programming Accessible Website and App UIs video
- ? Declarative REST: State Machines for the Web video
- Continuous Delivery for Architects - Neal Ford video
- ? High Performance in the Critical Rendering Path - how to make pages to load fast video
- This is Water - Neal Ford - an excursion into a strange, fantastical world with things like immutable database server, phoenix machines, and lambdas. video
- Securing Web APIs – Patterns & Anti-Patterns video
- ? Functional Data - event sourcing & FP video
- ? Taking other peoples money: A guide to online payments video
- ? Running Docker and Containers in Development and Production video
- Not Even Close: The State of Computer Security
- 595 billions income - untouched by human hands video
- Boosting security with HTTP headers video
- The rest of ReST - we'll look at the challenges of building usable real-world ReST APIs: Hypertext Application Language (HAL), HTTP Patch, ... video
- ? How do you scale a logging infrastructure to accept a billion messages a day? - DB -> ELK -> ELK + Kafka video
- ? Learning Client Hypermedia from the Ground Up - how to move specific knowledge of 1) addresses, 2) inputs, and 3) workflow out of the client app and place it into the message => a more robust, adaptable, and resilient client video
- ? Make it Faster - Lessons Learned from Benchmarking NoSQL on the AWS Cloud - best practices for performing database benchmarking on the AWS cloud & how to get more speed and efficiency in your production workloads video
- ? Crafting Evolvable Web API Representations - like structuring for evolution, sizing for optimum caching, the different ways to include metadata, ... video
- ? Mob Programming, A Whole Team Approach video
- ? Removing barriers - JetBrains's good and bad expericences with minimizing management video
Keynote Data and Goliath ☆☆☆☆
Why do companies fail at adopting Functional Programming?
According to the NDC Oslo talk Lean and Functional Programming by Bryan Hunter, these are the reasons why companies fail to adopt FP:
By Matthew Podwysocki
Lot of work (setup, remove listeners ...), not composable.
No way to abort promise in progress. (Me: has to remember to check for errors:
No try-catch-finally; only try-catch.
Node: Stream 1 were terrible (pause/resume unusable, data sent before ready, ...).
Example: Functional Reactive Programming decisively beats Imperative on simplicity, length
@theburningmonk Yan Cui has a nice example demonstrating how Functional Reactive Programming [slides 185 - 206] (with Elm's Signals) yields a much shorter and easier to understand (one you know FRP) code than an imperative code with mutations spread all over the code base.
Use the Up and Down keys to move the platforms and thus bounce the ball from left to right and back:
The imperative solution
Notes from Troy Hunt''s Hack Yourself First workshop
Highlights - resources
Personal security and privacy
- https://www.entropay.com/ - a Prepaid Virtual Visa Card
- mailinator.com - tmp email
- f-secure VPN
- https://www.netsparker.com/ - scan a site for issues (insecure cookies, framework disclosure, SQL injection, …) (lot of $k)
- https://report-uri.io/ - get reports when CSP rules violated; also displays CSP headers for a site in a human-friendly way
- https://securityheaders.io/ check quality of headers wrt security
- free SSL - http://www.startssl.com/, https://www.cloudflare.com/ (also provides web app firewall and other protections) ;
- SSL quality check: https://www.ssllabs.com/ssltest/
- https://letsencrypt.org/ - free, automated, open Certificate Authority (Linux Found., Mozilla)
- HSTS Preload - tell Chrome, FF that your site should only be ever loaded over HTTPS - https://hstspreload.appspot.com/
- https://twitter.com/jmgosney - one of ppl behind http://passwordscon.org . http://password-hashing.net experts panel. Team Hashcat.
AWS API: Proper syntax for filtering by tag name and value (e.g. describeInstances)
It took me quite a while to figure out the right syntax for filtering instances by tag name and value in the AWS EC2 API's describeInstances.
- Previous (2)Next (23)