Author Archives: Shine Technologies

A Deep Dive into DynamoDB Partitions

Posted on June 27, 2016 by Shine Technologies

Databases are the backbone of most modern web applications and their performance plays a major role in user experience. Faster response times – even by a fraction of a second – can be the major deciding factor for most users … Continue reading →

Posted in databases | Tagged , , , , | Leave a comment

Filling the BigQuery paddling pool from the Kinesis Hosepipe

Posted on June 21, 2016 by Shine Technologies

Introduction This blog post details how we solved the problem of analysing large amounts of HTTP request logs for one of our clients. Spoiler: we used Amazon’s Kinesis and Lambda to stream the data into Google’s BigQuery for analysis. Read on for … Continue reading →

Posted in Node.js | Tagged , , , , , , , | 3 Comments

The most important thing when picking HTTP status codes

Posted on June 3, 2016 by Shine Technologies

Every couple of months I’m in a meeting where a couple of developers start arguing about which HTTP status codes to use in their RESTful API, or where they decide to not use HTTP status codes at all and instead … Continue reading →

Posted in Opinion, Services | Tagged , , , | Leave a comment

Orchestrating Tasks Using AWS SWF

Posted on May 20, 2016 by Shine Technologies

Lately I’ve been doing a lot of work managing batch-processing tasks. Broadly speaking, there are 2 types of trigger for such a task: time-based and logic-based. The former can be easily done by cron/scheduled jobs. The latter can be a bit tricky, … Continue reading →

Tagged , , , | Leave a comment

The Emergence of The 3 Towers: DevSecOps

Posted on May 13, 2016 by Shine Technologies

I had the opportunity to attend AWS bootcamp in Sydney a couple of weeks ago. The session I chose to attend was entitled “Securing Cloud Workloads with DevOps Automation”. There were many interesting concepts discussed, all hinging around the new term ‘DevSecOps’. … Continue reading →

Posted in Continuous Delivery, Continuous Integration, DevOps | Tagged , , , | 1 Comment

Shine hosts a successful Digital Leaders Breakfast

Posted on April 15, 2016 by Shine Technologies

Yesterday Shine hosted a number of Digital Leaders to breakfast in Melbourne, aiming to share experiences and learning. More than 20 participants from organisations including Coles, Energy Australia, NAB, ANZ, Telstra, Fairfax Media and Australia Post heard presentations from Todd Copeland … Continue reading →

Posted in Uncategorized | Leave a comment

Creating a serverless ETL nirvana using Google BigQuery

Posted on March 30, 2016 by Shine Technologies

Quite a while back, Google released two new features in BigQuery. One was federated sources. A federated source allows you to query external sources, like files in Google Cloud Storage (GCS), directly using SQL. They also gave us user defined functions (UDF) in that … Continue reading →

Posted in databases, Javascript | Tagged , , , , , , , , , , , , | 4 Comments

Pablo rocking the stage at Google’s annual cloud event!

Posted on March 29, 2016 by Shine Technologies

Last week, Shine’s very own Pablo Caif gave a presentation at GCP Next 2016 in San Francisco, which is Google’s largest annual cloud platform event. Pablo delivered an outstanding talk on the work Shine have done for Telstra, which involves building solutions on the GCP … Continue reading →

Tagged , , , , , , , | Leave a comment

Shine’s Pablo Caif to present at GCP Next 2016!

Posted on March 1, 2016 by Shine Technologies

Shine is extremely proud to announce that Pablo Caif has been invited to present at GCP Next 2016, which is Google’s largest annual cloud platform event held in San Francisco. Pablo will be presenting on the work Shine have done for Telstra, which … Continue reading →

Posted in databases | Tagged , , , , , , , | Leave a comment

NoSQL in the cloud: A scalable alternative to Relational Databases

Posted on February 19, 2016 by Shine Technologies

With the current move to cloud computing, the need to scale applications presents itself as a challenge for storing data. If you are using a traditional relational database you may find yourself working on a complex policy for distributing your database … Continue reading →

Posted in databases, Uncategorized | Tagged , , , , , | Leave a comment