Posts
Misconfigured Kubeflow workloads are a security risk
“During April, we observed deployment of a suspect image from a public repository on many different clusters. The image is ddsfdfsaadfs/dfsdf:99. By inspecting the image’s layers, we can see that this image runs an XMRIG miner:” Source
Welcome to the cloud …
Check the running containers: The easiest way to spot an XMRIG miner in a Kubernetes cluster is by checking the running containers. Use the kubectl get pods command to get a list of all the running pods in the cluster.
read morePosts
dataflow real time + aggregate
A great way to split up your pipeline based on the urgency of results aggregate-data-with-dataflow
read morePosts
Calling Native Libraries from Java
A couple of options I’ve used and seen: Java Native interface - watch out for segfaults! Project Panama - early access GraalVM - still really new. zt-exec - call the native library as an external process remotetea - an old favourite if it’s legacy C++ code
read morePosts
Crypto - diy?
To create your own cryptocurrency, you will need to:
Create a blockchain. This is the underlying technology that will support your cryptocurrency. There are many different blockchain platforms available, such as Ethereum, Bitcoin, and EOS. Design your cryptocurrency. This includes deciding on the name, symbol, total supply, and distribution method. You will also need to create a mining algorithm. Create a wallet. This is where your cryptocurrency will be stored. There are many different wallets available, both hardware and software.
read morePosts
Google Cloud IAM Madness
After the recent GCP outage related to IAM, I found some odd behaviour with gsutil/gcloud. A script that had faithfully run for many months stopped working with:
ServiceException: 401 Anonymous caller does not have storage.buckets.list access to project xxxx
I tried recreating the service account key used for the operation with no luck. To fix the problem, I had to create a new bucket!
gsutil mb -b on -l us-east1 gs://my-awesome-bucket123ed321/
read morePosts
BigQuery ML and Vertex AI Generative AI
BigQuery ML and Vertex AI Generative AI (GenAI) are both machine learning (ML) services that can be used to build and deploy ML models. However, there are some key differences between the two services.
BigQuery ML: BigQuery ML is a fully managed ML service that allows you to build and deploy ML models without having to manage any infrastructure. BigQuery ML uses the same machine learning algorithms as Vertex AI, but it does not offer the same level of flexibility or control.
read morePosts
Cloud Billing Budget API in beta
You can finally set budgets via the API in GCP. This is a huge relief to all those org admins out there who have had to do this manually.
AND, hold on to your hats, there’s terraform support as well! Looks like Christmas came late….
data "google_billing_account" "account" { provider = google-beta billing_account = "000000-0000000-0000000-000000" } resource "google_billing_budget" "budget" { provider = google-beta billing_account = data.google_billing_account.account.id display_name = "Example Billing Budget" amount { specified_amount { currency_code = "USD" units = "100000" } } threshold_rules { threshold_percent = 0.
read morePosts
Flink Kubernetes operators
How I wish these operators had existed a few years ago when I was setting up Flink…
https://github.com/GoogleCloudPlatform/flink-on-k8s-operator
https://www.ververica.com/blog/google-cloud-platforms-flink-operator-for-kubernetes
read morePosts
Running Flink in Production
This is a great watch for those beginning their journey with Flink.
read more