rutabagas
Rutabagas Vol. 3: The Softest Ware: Is
-
What are probably appropriate attacks on assigning calamity to the hand-wavy concept of “human error”. Lorin strengthens his theory that human error probably doesn’t exist by considering complex, fault-tolerant systems like Amazon S3 incapable of receiving “human error” as apology or eulogy when there’s a hard drive failure.
-
WHAT. Secret guerrilla human rights interventions by the artist collective GALA Committee on Melrose fucking Place.
For three years, as the denizens of the Melrose Place apartment complex loved, lost, and betrayed one another, the GALA Committee smuggled subversive leftist art onto the set, experimenting with the relationship between art, artist, and spectator. The collective hid its work in plain sight and operated in secrecy. Outside of a select few insiders, no one—including Aaron Spelling, Melrose’s legendary executive producer—knew what it was doing.
- There are widely acknowledged “hard things” in software development, but the pith used to describe them usually falls short or flat for me. But in less than 100 words jenniferplusplus achieves serious depth:
It’s a little bit shitposty, but it’s 100% true. People think cache invalidation is hard because it has no right answer, so it’s a question of iterating until you get a right-enough answer.
People think naming things is hard because it’s about communicating with each other.
People don’t even think about managing dependencies. It’s invisible. It’s about cooperating across time and organizations to do maintenance chores. It’s so hard and so unglamorous that people can’t even see it.
What’s important to observe here is (the appropriate, warranted, overlooked) framing of these software problems as social problems, not logical puzzles or struggles with language and framework semantics. Even cache invalidation, despite being a requirement in nearly any useful application that nearly every developer must address, escapes a general solution – meaning developers must continuously arrive at socially acceptable, technically sufficient solutions – but not total – as the seasons change and the desires of forces and actors in the system change. There is always more than one person at large in negotiating the solution, if even only a developer and a single someone using the software.
- jenniferplusplus again, but this time bringing Peter Naur’s lasting insights into the AI conversation: programming is theory building. Software developers are quickly discovering the usefulness of LLMs trained on code to assist in some of their more menial daily work – especially when working with common constructs (languages, frameworks) that are personally unfamiliar – but it’s hard to imagine LLMs replacing the core intellectual work which requires understanding: like the socially syrupy hardness they discuss in the above toot. Although we might imagine a generative computer program giving semi-useful suggestions for the complex|complicated|sophisticated|confusing real-world development scenarios if trained on the correct historical inputs for that milieu; and if you were stuck in a vacuum somewhere without recourse to more senior colleagues who possess similar tokens of guidance to steer you toward the right away to handle background jobs (without such an exhaustive cataloging as you’ll see ChatGPT 3.5 vomit below):
Handling background jobs is a common requirement in software development, especially in scenarios where certain tasks need to be executed asynchronously or in the background to avoid blocking the main application thread. There are various ways to handle background jobs, and the choice often depends on the specific requirements and the technology stack being used. Here are some common approaches:
Task Queues:
Message Queues: Use message queuing systems like RabbitMQ, Apache Kafka, or Redis to enqueue and dequeue tasks. Popular libraries for this approach include Celery (Python), Sidekiq (Ruby), and Resque (Ruby).
Job Queues: Platforms like AWS Simple Queue Service (SQS) or Google Cloud Tasks can be utilized to manage queues of tasks.
Asynchronous Programming:
Async/Await: In languages that support asynchronous programming (e.g., Python with asyncio, JavaScript with Node.js), you can use async/await to perform tasks concurrently without blocking the main thread.
Multithreading/Multiprocessing: Use threads or processes to run background tasks concurrently with the main application. Be cautious with shared resources and synchronization.
Background Job Processing Frameworks:
Delayed Job Processing: Libraries like Sidekiq, Delayed Job (Ruby), or Hangfire (.NET) allow you to schedule and execute background jobs at a specified time or after a certain delay.
Job Scheduler: Tools like cron (Unix/Linux) or Windows Task Scheduler (Windows) can be used to schedule recurring background jobs.
Serverless Computing:
Functions as a Service (FaaS): Platforms like AWS Lambda, Azure Functions, or Google Cloud Functions enable you to run individual functions in the cloud without managing the underlying infrastructure.
Database-backed Queues:
Database Queues: Store tasks in a database table and have worker processes periodically check for and process these tasks. This approach can use a library like Active Job (Ruby on Rails) or Hangfire (C#).
Container Orchestration:
Kubernetes: Container orchestration platforms like Kubernetes can be used to deploy and manage background job processing containers.
Job Management Services:
Job Management Platforms: Use specialized services like Jenkins, CircleCI, or GitLab CI/CD for scheduling and executing background jobs.
Webhooks and Event-Driven Architectures:
Event-Driven: Design your system as an event-driven architecture, where background tasks are triggered by events. Webhooks can be used to notify the system of events.
Choose the approach that aligns with your application's architecture, scalability requirements, and the specific nature of the background tasks you need to handle. Keep in mind factors such as reliability, scalability, and ease of maintenance when making your decision.
Quoting Nauru’s seminal vintage paper (1985, the year my brother was born):
The fundamental task of software development is not writing out the syntax that will execute a program. The task is to build a mental model of that complex system, make sense of it, and manage it over time.
So anyway this post is both a big “thank you” to the many nerds who are chill and easygoing and joyful, and an exhortation to others to free yourself of contemptuous, competitive attitudes, and re-embrace the intellectual curiosity that presumably once brought you into programming in the first place.
Saturday December 16, 2023