360SOFTY

Insights

Engineering Insights

Practical writing on software architecture, SaaS products, AI automation, legacy modernisation, and the business of building reliable systems.

RSS

Curated links from external sources — not 360Softy original articles.

ExternalSoftware Engineering
Google Developers Blog

Accelerating on-device AI: A look at Arm and Google AI Edge optimization

Integration of Arm Scalable Matrix Extension 2 (SME2) and the Google AI Edge software stack enables high-performance, on-device generative AI by turning the CPU into a powerful matrix-compute accelerator. Using Stability AI’s "stable-audio-open-small" model as a case study, it outlines a streamlined "Convert, Optimize, and Deploy" pipeline that utilizes LiteRT, XNNPACK, and KleidiAI to automate hardware acceleration. The resulting implementation achieves over a 2x speedup in audio generation and

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Announcing Genkit Middleware: Intercept, extend, and harden your agentic apps

Genkit is an open-source framework designed to help developers build production-ready, agentic AI applications using TypeScript, Go, Dart, and Python. The framework utilizes a powerful middleware system that intercepts generation calls to inject custom behaviors like retries, model fallbacks, and human-in-the-loop tool approvals. By attaching hooks at the generate, model, and tool layers, developers can ensure high reliability and deterministic control over model outputs. Furthermore, Genkit all

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Build Long-running AI agents that pause, resume, and never lose context with ADK

How to transition from stateless chatbots to production-grade agents capable of managing long-running enterprise workflows, such as HR onboarding, that span days or weeks. It introduces the Agent Development Kit (ADK) and its architectural shifts, specifically using durable state machines and persistent session storage to ensure an agent never loses context during "idle time" or server restarts. By leveraging event-driven webhooks and multi-agent delegation, the tutorial demonstrates how to buil

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Supercharging LLM inference on Google TPUs: Achieving 3X speedups with diffusion-style speculative decoding

Researchers at UCSD have successfully implemented DFlash, a block-diffusion speculative decoding method, on Google TPUs to bypass the sequential bottlenecks of traditional autoregressive drafting. By "painting" entire blocks of candidate tokens in a single forward pass rather than predicting them one-by-one, the system achieved average speedups of 3.13x, with peak performance nearly doubling that of existing methods like EAGLE-3. This open-source integration into the vLLM ecosystem optimizes TPU

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Building with Gemini Embedding 2: Agentic multimodal RAG and beyond

Google has announced the general availability of Gemini Embedding 2, a unified model that maps text, images, video, audio, and documents into a single semantic space. This model allows developers to process interleaved multimodal inputs in a single request, significantly improving performance for tasks like agentic RAG, visual search, and content moderation. By supporting over 100 languages and offering features like task-specific prefixes and Matryoshka dimensionality reduction, the model provi

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Speeding Up AI: Bringing Google Colossus to PyTorch via GCSFS and Rapid Bucket

Google Cloud has introduced a high-performance integration that connects Rapid Storage directly to PyTorch via the fsspec interface to eliminate AI training bottlenecks. By utilizing Google’s Colossus architecture and bidirectional gRPC streaming, the solution offers up to 15 TiB/s aggregate throughput and significant reductions in latency. These improvements allow developers to speed up total training time by 23% with zero code changes required beyond updating the storage bucket type.

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Building real-world on-device AI with LiteRT and NPU

LiteRT is a production-ready framework designed to help mobile developers unlock the power of Neural Processing Units (NPUs), overcoming the performance and battery limitations of traditional CPU or GPU processing. By providing a unified API that abstracts away hardware complexities, it allows industry leaders like Google Meet and Epic Games to deploy sophisticated AI models for real-time video, animation, and speech recognition with significantly higher efficiency. The platform further supports

Google Developers BlogRead original
ExternalSoftware Engineering
Google Developers Blog

Agents CLI in Agent Platform: create to production in one CLI

Google Cloud has introduced the Agents CLI, a specialized tool designed to bridge the gap between local development and production-grade AI agent deployment. The CLI provides coding assistants with machine-readable access to the full Google Cloud stack, reducing context overload and token waste during the scaffolding process. By streamlining evaluation, infrastructure provisioning, and deployment into a single programmatic backbone, the tool enables developers to move from initial concept to a l

Google Developers BlogRead original

Work with 360Softy

Building a SaaS product, AI system, or business platform?

Book a free consultation and we will tell you honestly whether we can help.

Book a consultationView our services →