Tag Archives: aws

A Deep Dive into DynamoDB Partitions

Posted on June 27, 2016 by Shine Technologies

Databases are the backbone of most modern web applications and their performance plays a major role in user experience. Faster response times – even by a fraction of a second – can be the major deciding factor for most users … Continue reading →

Posted in databases | Tagged , , , , | Leave a comment

Filling the BigQuery paddling pool from the Kinesis Hosepipe

Posted on June 21, 2016 by Shine Technologies

Introduction This blog post details how we solved the problem of analysing large amounts of HTTP request logs for one of our clients. Spoiler: we used Amazon’s Kinesis and Lambda to stream the data into Google’s BigQuery for analysis. Read on for … Continue reading →

Posted in Node.js | Tagged , , , , , , , | 3 Comments

Orchestrating Tasks Using AWS SWF

Posted on May 20, 2016 by Shine Technologies

Lately I’ve been doing a lot of work managing batch-processing tasks. Broadly speaking, there are 2 types of trigger for such a task: time-based and logic-based. The former can be easily done by cron/scheduled jobs. The latter can be a bit tricky, … Continue reading →

Tagged , , , | Leave a comment

The Emergence of The 3 Towers: DevSecOps

Posted on May 13, 2016 by Shine Technologies

I had the opportunity to attend AWS bootcamp in Sydney a couple of weeks ago. The session I chose to attend was entitled “Securing Cloud Workloads with DevOps Automation”. There were many interesting concepts discussed, all hinging around the new term ‘DevSecOps’. … Continue reading →

Posted in Continuous Delivery, Continuous Integration, DevOps | Tagged , , , | 1 Comment

NoSQL in the cloud: A scalable alternative to Relational Databases

Posted on February 19, 2016 by Shine Technologies

With the current move to cloud computing, the need to scale applications presents itself as a challenge for storing data. If you are using a traditional relational database you may find yourself working on a complex policy for distributing your database … Continue reading →

Posted in databases, Uncategorized | Tagged , , , , , | Leave a comment

Video of ‘Big Data the Cloud Way’ presentation now available

Posted on August 7, 2015 by Shine Technologies

Video of the recent ‘Big Data the Cloud Way’ presentation by Shine’s Graham Polley and Shane Neubauer is now available. In it, Graham and Shane talk about how they used both the Google and Amazon cloud stacks at Telstra to … Continue reading →

Tagged , , , , , , | Leave a comment

Upcoming Shine Event: Leveraging Big Data in the Cloud using Google and AWS

Posted on June 18, 2015 by Shine Technologies

Shine would like to invite you to a special event we are hosting in Melbourne on July 8. In a one-off presentation to be held at ACMI in Federation Square, Shiners Graham Polley and Shane Neubauer will share with you … Continue reading →

Tagged , , , , , , , | Leave a comment

Building a file explorer on top of Amazon S3

Posted on May 20, 2015 by Shine Technologies

Amazon S3 is a simple file storage solution that is great for storing content, but how well does it stack up when used as the storage mechanism for a web-based file explorer? Recently I was tasked with doing just this … Continue reading →

Tagged , , | 4 Comments

Battling it out for glory at Battlehack

Posted on February 13, 2015 by Shine Technologies

WHAT A WEEKEND! I’m still recovering from the 24 hours of coding and the unhealthy amount of coffee and snacks I’ve ingested during this hackathon. Two weeks ago, Melbourne was host to the first Battlehack of 2015. “What’s a Battlehack?” … Continue reading →

Posted in Javascript | Tagged , , , , , , , , , , , | 1 Comment

Getting New Relic and RDS to play nice

Posted on November 28, 2014 by Shine Technologies

Anyone who’s ever had to support server infrastructure of any kind knows the value of having a comprehensive, automated monitoring solution in place. With this in mind, we have begun to roll out the New Relic platform to monitor all … Continue reading →

Posted in DevOps, Tools | Tagged , , , , , | 1 Comment