Google Kubernetes Engine (GKE) is a popular container orchestration platform that allows developers to deploy and manage containerized applications at scale. However, a recent security vulnerability has been discovered in GKE that could allow attackers to gain access to clusters and steal data or launch denial-of-service attacks. The vulnerability is caused by a misunderstanding about the system:authenticated group, which includes any Google account with a valid login. This group can be assigned overly permissive roles, such as cluster-admin, which gives attackers full control over a GKE cluster.

AWS App Runner and Google Cloud Run are two serverless computing platforms that can help you deploy and run containerized applications without having to worry about servers. Both platforms are relatively new, but they have quickly become popular choices for developers. What are the similarities? Both platforms are serverless, meaning that you don’t have to provision or manage servers. The platforms will automatically scale your application up or down based on demand, so you only pay for the resources that you use.

Azure networking and GCP networking are both comprehensive cloud networking services that offer a wide range of features and capabilities. However, there are some key differences between the two platforms. Azure networking offers a more traditional networking model, with a focus on virtual networks (VNets), subnets, and network security groups (NSGs). VNets are isolated networks that can be used to group together resources, such as virtual machines (VMs), storage, and applications.

It can be tough to try to reverse engineer an existing project that has never used terraform. Terraformer can look at an existing project and generate the corresponding terraform code for you. I tried it out on an existing legacy project which used Google Cloud Storage, BigQuery and various service accounts. The setup was a little tricky so I put together a script to simply things. The script assumes you have gcloud setup or a service account key/impersonation and you may need to adjust the –resources parameter.

One hour ago: bq cp mydataset.table@-3600000 mydataset.table_restored Absolute (ms since UNIX epoch) GMT: Wednesday, 26 May 2021 13:41:53 = 1622036513000 https://www.epochconverter.com/ bq cp mydataset.table@1622036513000 mydataset.table_restored More on Bigquery time travel

Feature Confluent Cloud Kafka Google Cloud Pubsub Notes Data Retention Set retention per topic in Confluent Cloud, including unlimited retention with log compaction. Retains unacknowledged messages in persistent storage for 7 days from the moment of publication. There is no limit on the number of retained messages. Have to write custom subscriber/publisher to save beyond 7 days [L] + ongoing BAU [S] Replay A consumer request an “offset”, however the retention period is dictated by the broker config “Snapshots” can be created for later replay by these are limited to 7 days as per retention policy.

A great way to split up your pipeline based on the urgency of results aggregate-data-with-dataflow

After the recent GCP outage related to IAM, I found some odd behaviour with gsutil/gcloud. A script that had faithfully run for many months stopped working with: ServiceException: 401 Anonymous caller does not have storage.buckets.list access to project xxxx I tried recreating the service account key used for the operation with no luck. To fix the problem, I had to create a new bucket! gsutil mb -b on -l us-east1 gs://my-awesome-bucket123ed321/

You can finally set budgets via the API in GCP. This is a huge relief to all those org admins out there who have had to do this manually. AND, hold on to your hats, there’s terraform support as well! Looks like Christmas came late…. data "google_billing_account" "account" { provider = google-beta billing_account = "000000-0000000-0000000-000000" } resource "google_billing_budget" "budget" { provider = google-beta billing_account = data.google_billing_account.account.id display_name = "Example Billing Budget" amount { specified_amount { currency_code = "USD" units = "100000" } } threshold_rules { threshold_percent = 0.

An invaluable start on how to start managing GCP projects with Terraform. I wish I’d found this a year ago.

Rather than fully configuring your backend.tf in a file.``` terraform { backend “gcs” { bucket = “my-bucket-123” prefix = “terraform/state” } I prefer to use the command line in order avoid polluting the code with any environment specific names. terraform init \ -backend-config=“bucket=my-bucket-123” \ -backend-config=“prefix=terraform/state

I’m glad Google are finally starting to embrace Terraform by creating their own modules. Version 0.1.0 of the project-factory looks really promising.

I’m currently bench-marking Flink against Google Cloud Dataflow using the same Apache Beam pipeline for quantitative analytics. One observation I’ve seen with Flink is the tail latency associated with some shards. Google Cloud Dataflow can optimise away stragglers in large jobs using “Dynamic Workload Rebalancing". As far as I know, Flink is currently unable to perform similar optimisations.