You are right and what some people miss is that social engineering being the vector to gain foothold doesn’t mean that it was sufficient to allow the breach. Almost always you need some other weakness (or a series of them). Except when the weaknesses are so had that you don’t need a foothold at all (like this case), or when the social engineering gives you everything (rare, but you might convince you someone to give you access to data etc.).

A whole separate conversation is deserved by how effective (or not) social engineering training is. Quite a few good papers about the topic came out in the last fee years.

Social/Political problems need social/political solutions, not technical solutions.

But you know what the kernel is. You know that syscalls are a thing, you know what role the kernel performs, you know that different filesystems have different properties (and pros and cons), etc…

You don’t need to know the details, perhaps, but you can’t ignore the fundamental theoretical concepts of kernel and OS. You might not know the whole detail of the boot procedure, but if your machines are stuck on boot, you know at least what to look for.

Here I was talking about equally foundational topics. There is nothing “above” - say - producing attestations and then verifying them. That’s literally all there is to it, but if you don’t understand the theory behind it, what exactly are you doing? As as I said, I don’t care about the details, I didn’t expect someone mentioning ciphers or timestamp authorities, transparency logs etc. All I would expect is “we produce a signature with a bunch of metadata and we verify it where we consume the artifact, so that we are sure that the artifact has the properties attested by the signature”.

Not knowing this is like someone claiming that they administer Linux machines but can’t explain what network interfaces are or how routing is determined. This is not a question of being expert on different layers, this is just being oblivious to those other layers completely.

A cloud VM, just shut it down and you’re done.

If this flexibility is needed, and it’s an “if”, a dedicated server does the same. But even a cloudVM is already lower level compared to other services (which are even more abstract) - like EKS, SQS, etc.

The value an organization provides to customers should be the primary focus of the business, the rest is a means to sharpen that focus.

In my experience this often translates in values that flows to AWS, while the company giving value to customers is stuck with millions of cloud bills each month, and a large engineering footprint that eventually needs to cut, leaving fewer and fewer people working on the product.

That said, I acknowledge that cloud has business reasons to exist, I wrote an entire other post about my hate for it, but I still acknowledge that. However there are some myths that finally are getting dispelled (outsource infra and focus on your product).

I mean, the person in question had “hardening EKS” on their CV. EKS still means that the whole data plane is your responsibility. How can you harden a cluster without understanding the foundation of container security (isolation primitives, capabilities, etc.)? Workload security is very much part of the job.

I mean the moment some pod will need to run with some privilege (say, a log forwarder which gets host logs), and you need to “harden” the cluster, what do you do if you don’t understand the concept of capabilities? I will tell you what, because I asked this very question, and the answer was “copy the logs elsewhere”, which is the “make it work with the hammer solution” that again shows the damage of not understanding.

I am with you about different scopes, skillsets etc. But here we were interviewing people with a completely matching skillset on paper.

Thanks, indeed I think there are many parallels with other areas. I will check it out.

I totally agree with you, but I don’t think this is the specific case. Most of the rejections in our case (which I can see) on the preliminary screening were based on lacking CV skills. Which is stupid in its own way, but at least makes sense assuming we are looking for those skills specifically.

For the rest, the company is a remote company paying good salaries for the European market, I would say slightly above market average in many metrics.

I will sift more into the rejections, but from what I have seen, almost all those who had the screening phone call made it to the interview (I.e., rejections were mostly cv-based).

But those are absolutely not the only 2 levels. Server rental can be managed easily by the same infra team who manages the cloud, for a fraction of cost.

I will say more, the same exact team that spends time managing EKS clusters could manage self-managed clusters and have money to spare for additional hires.

Mind you that my take and experience is specifically in the context of security.

I struggle to make the parallel that you suggest (which might work for some areas) with a security engineer.

Say, a person learned to brainlessly parrot that pods need to have setting x or z. If they don’t understand them, they can’t offer meaningful insight in cases where that’s not possibile (which might be specific), they can’t provide a solid risk analysis etc.

What is the counterpart to this gap? Because I struggle to see it. Breadth of areas where this superficial knowledge is available is useless, IMHO.

Ahaha yes, that might be the case, but I started to lose hope if the top of the applicants (out of hundreds of rejected!) all exhibits this behavior. I can’t help but feel that now we are looking for people with a mindset and skillset that is simply disappearing in the industry.

And as I said in another post, I perfectly acknowledge that if I stopped reading and investigating stuff on my own, I could absolutely keep my job by just mindlessly administering a few services and rephrasing CIS benchmarks…

This is quite a trite argument from my point of view. Also, this is from the perspective of the business, which I don’t particularly care about, and I tend to look from the perspective of the worker.

Additionally, the cloud allows to scale quickly, but the fact that it allows to delegate everything is a myth. It’s so much a myth that you see companies running fully on cloud with an army on people in platform teams and additionally you get finops teams, entire teams whose job is optimizing the spend of cloud. Sure, when you start out it’s 100% reasonable to use cloud services, but in the medium-long term, it’s an incredibly poor investment, because you still need people to administer the cloud plus, you need to pay a huge premium for the services you buy, which your workforce now can’t manage or build anymore. This means you still pay people to do work which is not your core business, but now they babysit cloud services instead of the actual infra, and you are paying twice.

Cloud exploded during the times of easy money at no interest, where startups had to build some stuff, IPO and then explode without ever turning a single dollar of profit. It’s a model that fits perfect in that context.

Not when the skillset is essentially outsourced and you are left consuming the product of that skillset.

Understanding is nonnegotiable in security, IMHO.

You can’t fail to understand how signature attestation works, if you are implementing it, to make one example I made in the post. Otherwise you end up verifying the signature in the CI (like that person claimed it should be done) and waste the whole effort. You can definitely still outsource the whole infra and scripting to Github, but you still need to understand. The problem is that when you can outsource everything, at some point understanding becomes an extra step.

That’s the thing! I think it wouldn’t be conceivable that your “principal engineer” (real position for one of the people) doesn’t understand the basic theory of the stuff they are implementing. Now it feels you can instead work years and years just shuffling configuration and pressing buttons, leading to “senior” people who didn’t gather actual years of experience.

I don’t want to pretend I am outside this logic. I am very much part of this problem myself, having started my career 10 years ago. I do despise cloud services though (if anything, they are super boring), so I tend to work with other stuff. But I could 100% just click buttons and parrot standard and keep accruing empty years of experience…