Archive for the ‘Worst Practices’ Category

Your Security Needs to Work Together

Monday, January 2nd, 2017

Here follows a short analogy regarding the importance of getting all your security to work together.

Let us say that a bank is under attack by a gang of thieves that intend to break into its vault and walk away with what’s inside it. Not as sophisticated as getting one of their members to be promoted to a C-level position and then engage in massive securities fraud, but it’ll do for our analogy.

Let us say also that the bank has a security force to deal with each aspect of physical security, but they do not cooperate with each other. The thieves attack in broad daylight, after they’ve had a nice lunch and a nap, so they’re absolutely fresh.

At 1500, the guard watching the video surveillance system notes in his log that ten masked men with weapons entered the bank. He knows this will be something for his team to discuss in their weekly meeting in two days.

At 1501, the guards in charge of the bank lobby note the ten armed men heading towards the vault door. As they are not robbing any tellers in the lobby, the lobby security men do not interact with the men in masks.

At 1503, the guards in charge of the vault door respond to an alert that the door has been blown off its hinges and is laying on the floor of the bank vault. With crack speed, they remove the old door and replace it with a new one. A new guy on the team asks what to do about the ten guys with masks that are empyting out the deposit boxes, but the team lead tells him not to worry, the deposit box guys will handle that: they just need to focus on the door.

At 1505, the guards in charge of the deposit boxes arrive at the vault door, but they cannot enter, as they do not have access.

At 1534, the guards in charge of the deposit boxes note that they can now enter the vault area, as the door has been blown off its hinges. They pass by ten men in masks, each with a weapon and a large bag that seems to be full to capacity. As the men are leaving the vault area, that is not their security concern.

At 1535, the guards in charge of the vault door respond to another alert that a door has been blown off its hinges. It’s a good thing that they always keep two spare vault doors in stock! The team lead prepares the two blown-up doors for an RMA to their manufacturer.

At 1536, the guards in charge of the lobby note the ten men in masks, en route out of the bank front door. Again, they pose no immediate threat to the tellers, so there is no call to engage with them.

At 1538, the guard in charge of the security cameras notes in his log that ten suspicious-looking men in masks with large, full bags, were making their way to their cars in the parking garage. This will be another interesting thing to discuss at the security camera guard team meeting, exactly the sort of thing they should be noting in their logs.

At 1545, the guards in charge of the vault door are wondering what to do with a request for access out of the vault from the security deposit box team. How did they manage to get into the vault without anyone having authorized their entry? The vault door team lead plans to write up the security deposit team for an access violation, as soon as he’s finished with the vault door RMA paperwork. 


Monday, September 19th, 2016

I had a very sad friend. His company bought all kinds of really cool stuff for security monitoring, detection, and response and told him to point it all at the firm’s offices in the Russian Federation. Because Russia is loaded with hackers, right? That’s where they are, right?

Well, he’d been running the pilot for a week and had nothing to show for it. He knows that the tools have a value, and that his firm would benefit greatly from their widespread deployment, but he’s worried that, because he didn’t find no hackers nowhere in the Hackerland Federation, his executives are going to think that these tools are useless and they won’t purchase them.

So I asked him, “Do you have any guidance from above on what to look for?”

“Hackers. They want me to look for hackers.”

“Right. But did they give you a software whitelist, so that if a process was running that wasn’t on the list, you could report on it?”

“No. No whitelist.”

“What about a blacklist? Forbidden software? It won’t have everything on it, but it’s at least a start.”

“Yes, I have a blacklist.”

“Great! What’s on it?”

“Hacker tools.”

“OK, and what are listed as hacker tools?”

My friend sighed the sigh of a thousand years of angst. “That’s all it says. Hacker tools. I asked for clarification and they said I was the security guy, make a list.”

“Well, what’s on your list?”

“I went to Wikipedia and found some names of programs there. So I put them on the list.”

“And did you find any?”

“Some guys are running the Opera browser, which has a native torrenting client. I figured that was hacker enough.”

Well, security fans, that’s something. We got us a proof of concept: we can find active processes. I described this to my friend, and hoped that he could see the sun peeking around the clouds. But it was of no help.

“They’re not going to spend millions on products that will tell them we’re running Opera on a handful of boxes!”

He had a point, there. Who cares about Opera? That’s not a hacker tool as featured on the hit teevee show with hackers on it. And, to be honest, the Russian offices were pretty much sales staff and a minor production site. The big stashes of intellectual property and major production sites were in the home office, in Metropolis, USA.

So I asked, “Any chance you could point all that stuff at the head office?”

“What do you mean?”

“Well, it’s the Willie Sutton principle.”

“Who was Willie Sutton?”

I smiled. “Willie Sutton was a famous bank robber. His principle was to always rob banks, because that’s where the money was. Still is, for the most part. Russia in your firm is kind of like an ATM at a convenience store. There’s some cash in it, but the big haul is at the main office. Point your gear where the money is – or intellectual property – and see if you don’t get a lot more flashing lights.”

My friend liked that. He also liked the idea of getting a software whitelist so he’d know what was good and be able to flag the rest as suspect. He liked the idea of asking the execs if they had any guidance on what information was most valuable, so that he could really take a hard look at how that was accessed – and who was accessing it.

And maybe there were tons of hackers in Russia, but they weren’t hacking anything actually in Russia. And maybe said hackers weren’t doing anything that was hacking-as-seen-on-television. Maybe they were copying files that they had legitimate access to… just logging on, opening spreadsheets, and then doing “Save As…” to a USB drive. Or sending it to a gmail account. Or loading it to a cloud share…

The moral of the story is: If your security policy is driven by the popular media, you don’t have a security policy.

The Internet of Things with Pre-Installed Backdoors

Friday, August 12th, 2016


The SEO-friendly URL says it all. The Rockwell PLCs in question have a RW SNMP community common to a range of their devices, undocumented, but if you can find it, you can light up every one of them.

Correction: there’s another SNMP string that allows even more access, also undocumented. That’s what’s makes this newsworthy. Not one backdoor, that’s old news. Two pre-installed backdoors, now we got us a story!

If you work with PLCs, read the article above, check to see if you’re using any of them, and then contact the manufacturer. You need to get all over this like a donkey on a waffle.

The Internet of Things Still in Development

Thursday, August 11th, 2016

So, I just read an article on some solar panels with a wifi connection that had a default admin name and password. That’s not news. What is news is that the solar panels were still in development and were mistakenly shipped to customers.

As we plunge deeper into the world of everything having an IP address, this incident highlights a new concern: what impact does improper labeling have on security? What if these were autonomous vehicles (our favorite bugbear) shipped to a dealer? More terrifying, what if these were controls for a LNG terminal or a nuclear reactor? It’s bad enough we have default credentials on production devices, but now we have to consider a mis-shipment of even less secure development devices.

Or, we can start to say “no”. The promises of cost savings and higher productivity need to be placed against a realistic risk assessment. Is saving a few bucks per IP-enabled lightbulb worth the possibility of a major PCI breach? OK, maybe I’m engaging in hyperbole, as well, but it’s no worse than the hyperbole of IoT marketers that aren’t telling the full story of how human fallibility is always a constant, even when we use computers to speed our poor decision-making processes.

We’ve had product recalls before, and we’ll have them again. But IoT ubiquity means a window of opportunity between the zero-day and the day of repair to wreak havoc, mayhem, and unintended accidents.

I’ll raise another concern: what about device interoperability? I know that if I have medication A, I may have to abstain from substance B if I don’t want a horrendous drug interaction. When will we be able to look at IoT devices working with each other and possibly breaking code as a result of such interoperation?

We need to have a Serious Discussion of Things before we have an Internet of Things.

A Night at the Outsourcer

Friday, August 5th, 2016


Driftwood: All right. It says the, uh, “The first part of the party of the first part shall be known in this contract as the first part of the party of the first part shall be known in this contract” – look, why should we quarrel about a thing like this? We’ll take it right out, eh?
Fiorello: Yeah, it’s a too long, anyhow. (They both tear off the tops of their contracts.) Now, what do we got left?
Driftwood: Well, I got about a foot and a half.

After talking with people from companies whose experiences with their outsourcing‍ contracts can be best described as “disappointing”, I wonder if they didn’t have the equivalent of the‍ Marx Brothers‍ representing them in their contract negotiations. I’m not saying that the corporate lawyers were idiots‍ , just that they may have been outclassed by the outsourcers’ lawyers. This is a specialized situation, after all.

Like the company doing the outsourcing, the outsourcer wants to maximize profits. Outsourcers are not charitable organizations, offering up low-cost business services to help the hapless firm with IT‍ needs. They want to get paid, Jack! Some may want a long-term, quality relationship with a client, but there are plenty out there that want to sign a contract that, on the surface, looks like it will reduce costs, but it contains hidden standard business practices‍ that will rake the clients over the coals.

One of the biggest gotchas in an outsourcing contract is the fact that the relationship between a company and its IT is no longer one of company to employee, but company to contractually provided service. That means the “one more thing” that managers like to ask for from their employees isn’t an automatic wish that will be granted. Did the contract authorize that one more thing? No? Well, that will cost extra, possibly a lot extra.

Another loss is the ability to say, “I know that’s what I wrote, but what I meant was…” as a preface to correcting a requested change. In-house staff can be more flexible and adapt to the refinement of the request. Outsourced staff? Well, it seems as though the staff were engaged to make a specific change, so there’s a charge for that, even though you decided to cancel the change in the middle of it. Now, the change you requested needs to be defined, submitted, and approved in order for us to arrange staff for the next change window…

There’s also the limit on the time-honored technique of troubleshooting the failed change and then making the troubleshooting part of the change. Consider a firewall change and then discovering that the vendor documentation left out a port needed for the application to work. In-house staff have no problem with adding that port and making things work. Outsourcers? If that change isn’t in writing, forget about it until it is. And, then, it may be a matter of rolling back the change and trying again, come the next change window.

Speaking of firewalls, that brings me to the “per line of code” charge. If the contract pays by the line of code, prepare for some bulky code if the contract does not explicitly state that lines of code must be consolidated whenever possible in order to be considered valid and, therefore, billable. Let me illustrate with an example.

My daughter is 14 and has zero experience with firewall rules. I asked her recently how many rules would be needed for two sources to speak to two destinations over five ports. She said five rules would be needed. I then gave a hint that the firewall help file said that ports could be grouped. Then, she proudly said, “one!”

While that’s the right answer for in-house IT staff, it’s the wrong answer for an outsourcer being paid by the line. 20 is the right answer in that case. It blew her mind when I told her how many different firms I’ve heard about that had 20 rules where one would do. As a teenager with a well-developed sense of justice, she was outraged. So long as contracts are signed that don’t specify when, how, and what to consolidate, she will continue to be outraged.

I didn’t have the heart to tell her about how some outsourcers contract to provide services like email, but the contract did not outline all the things we take for granted as part of email but which, technically, are not email. Shared calendars? Not email. Permissions for an admin assistant to open a boss’ Inbox? Not email. Spam filtering? Not email. Email is the mail server sending/receiving to other mail servers and allowing clients to access their own inboxes. Everything else is not email, according to the outsourcers’ interpretation of the contract. Email is just one example, and all the other assumptions made about all the other services add up with the above to create a situation in which the outsourcing costs significantly more than keeping the work in-house.

This can have significant impact on security. Is the outsourcer obligated to upgrade devices for security patching? Is the outsourcer obligated to tune security devices to run optimally? Is the outsourcer required to not use code libraries with security vulnerabilities? If the contract does not specify, then there is zero obligation. Worse, if the contract is a NoOps‍ affair in which the customer has zero visibility into devices or code, then the customer may never know which things need what vulnerabilities mitigated. There may be a hurried, post-signing negotiation of a new section about getting read rights on the firm’s own devices and code… and that’s going to come at a cost.

Another security angle: who owns the intellectual property in the outsourcing arrangement? Don’t make an assumption, read that contract! If the outsourcer owns the architecture and design, your firm may be in for a rough ride should it ever desire to terminate the contract or let it expire without renewing it.

I’m not even considering the quality of work done by the outsourcer or the potential for insider threat – those can be equal concerns for some in-house staff. The key here is that the contract is harsh, literal, and legally binding. That means vague instructions can have disastrous results. Tell an outsourcer to “make a peanut butter and jelly sandwich,” do not be surprised if the outsourcer rips open a bag of bread, smashes open the jars of peanut butter and jelly, mashes the masses of PB & J together, shoves the bread into that mass, and then pulls out the bread slices with a glob of peanut butter, jelly, glass, and plastic between them. He gave you what you specified: it’s not his fault that the instructions were vague.

There can be a place for oursourcing, particularly as a staffing solution for entry-level positions with high turnover. But every time I talk with someone from a place that either is currently in or is recovering from an outsourcing contract that went too far, I hear the horror stories. The outsourcers’ lawyers know what they’re doing and the firm’s lawyers fail to realize how specific they have to be with the contract language to keep from looking like they may as well have been the Marx Brothers‍.

Driftwood (offering his pen to sign the contract): Now just, uh, just you put your name right down there and then the deal is, uh, legal.
Fiorello: I forgot to tell you. I can’t write.
Driftwood: Well, that’s all right, there’s no ink in the pen anyhow. But listen, it’s a contract, isn’t it?
Fiorello: Oh sure.
Driftwood: We got a contract…
Fiorello: You bet.

Republican Party: Ur DOIN IT WRONG

Wednesday, July 27th, 2016

Well done, Rince Priebus. The chairman of the Republican National Committee (RNC) said, “Maybe our folks are better at securing our e-mail and our cloud and our data than the DNC. I don’t know what the answer to that is, Andrea, but at this point, we haven’t been hacked… but, I can assure if someone hacked my e-mails, they wouldn’t find me calculating against particular candidates and it’s not something that I would do.”

He could have saved some effort by simply saying, “We are pleased to announce a hackathon, starting now, directed against the RNC servers. While I doubt I have anything embarrassing on them, please feel free to share whatever you find with public sources of shared information including, but not limited to WikiLeaks, Pastebin, and an open Dropbox folder.” Or, for the even shorter translation that would produce the same invitation as in the above two comments: “ALL UR h4x R WEAKSAUSE!!!!@!~~~!!~~~!!!!111!!!eleven!!!”

Organizations need to have some defined style codes and talking points when representatives are speaking about security issues. Foremost among such codes and points should be an admonition to not tempt fate by declaring invulnerability to attacks or by saying there’s nothing worth finding on one’s network. Now, the people already attacking the RNC network are about to be joined by other, previously unmotivated individuals who now, out of a sense of curiosity or self-righteousness, are going to see if the RNC’s servers are indeed better secured and/or have nothing of value on them. Such information would then be shared, most likely on one of the Internets or maybe even a website, because Internets and websites are things hackers will use in their cyber.

I wrote that last sentence in jest, but it pains me to think that there are people in the RNC as well as the DNC, and a number of other organizations, that would have taken notes on that sentence if I presented it to them in a PowerPoint. I would then be asked follow up questions to clarify what is meant by “Internets”, “websites”, and “cyber.” Internet security is so much more than just looking both ways before crossing the street. It also involves not standing defiantly in the intersection while yelling “COME AT ME, BRO!” to approaching drivers.

Come to think of it, that would make another really cool slide. If you’re in the RNC or DNC, call me. I got a slide deck that will open your eyes!

This News Just in…

Friday, July 22nd, 2016

If you’re an infamous person that the government is out to get, don’t use iTunes. Or Facebook. The guy supposedly in charge of Kickass Torrents used both, his IP address for iTunes matched the IP address for editing his website’s Facebook page, and it was all over for him. He would seriously have benefited from adversary-resistant computing and adversary-resistant networking.

IT Personnel Areas

Thursday, October 17th, 2013

Most companies get this right, but there are a few outliers that haven’t gotten with the program. I’m talking about the right way to house your IT personnel at the workplace. Some people get it wrong and put their IT staff into a converted storage area.

Ideally, IT staff should go into an actual, unconverted, storage area. Just run an extension cord with a power outlet strip attached to the area and make sure they wear hard hats, if appropriate. If, however, you have insufficient warehouse space to allow your IT staff to office there, then you’ll have to get creative.

Inspect your building blueprints. You’ll notice that nearly everything will have a label. If you’re lucky, there will be a room without a label on it that’s next to the elevators and/or stairwell. It has a door and, when you see it in person, it’s half-full of building supplies and/or disused computer equipment. That’s the perfect spot for your IT guys!

If you don’t have that, don’t give up. You can still find them a suitable location. See if there’s an internal office – no windows at all on this one – that has very poor ventilation. You’re looking for a place that will either freeze or roast your staff, regardless of season, preferably with some airflow pattern that concentrates environmental evaporates – like 4-PC and Styrene from the latex backing in carpets – in that area. If you have more than one such room, pick the one that is furthest away from the data center and then be sure to use the one(s) that are closer to the data center for furniture storage.

If, for legal reasons, you have to provide a safe and tolerable work environment for your IT staff, there are still ways to optimize their work environment, even if you can’t encase them in a storeroom sarcophagus. If you have a satellite campus, removed from the main data centers, put them there. If there are other departments there, be sure to have your IT staff in their own section, as far removed from the amenities of the building as possible. If you do not have a satellite campus and you can’t stick them in a storeroom, it’s time to talk to a commercial real estate guy and get yourself a remote facility for your IT crew. If you can get your IT guys into a metal building in an industrial zone, that’s almost as good as a warehouse. Failing that, the far end of a light industrial park is another good spot.

Around the world, these are the kinds of environments IT people are used to. These are the environments they expect. If you actually give them windowed offices with close proximity to the data center, they will become disoriented and confused by their surroundings, and those stresses can lead to your IT staff losing their ability to lash out against passers-by. Should your IT staff acquire “people skills,” they’ll never get their work done as a result of having cheery interactions with other people. For them to be focused on their demanding tasks, they need to be kept in hellish, semi-barbaric environments, so that their only solace comes from fixing technological issues and vendor lunches.

How to Think Like a (bad) CIO

Monday, October 14th, 2013

1. Go to a magazine, blog, or website that purports to summarize IT information for executives. Alternatively, go to an airport men’s room.

2. Look at all the pictures, captions, and words in ads printed in large fonts. If in the airport men’s room, take note of whatever is advertised at eye level above the urinals.

3. Become obsessed with leveraging next-generation synergies with the emerging technology described, above.

4. Take the marketing to the next level. One way to do that is the expansion of benefits ad infinitum: if one of those devices/software platforms is good for a company, then one per user ought to be awesome. For example, if one firewall will protect a company, then one firewall per user will provide awesome protection. Another example: if one load-balanced virtual server cluster is good for a company, then getting every user his or her own load-balanced virtual server cluster will definitely deliver those leveraged next-generations synergies.

5. Present the next-level solution to your IT team and express confidence that they can get the implementation done right.

If you can do this and you are an IT professional, you will be ahead of the curve when the orders come down from above, and you’ll be ready to roll with the project.

PROTIP: having budget numbers ready to go on the outlandish ideas is a great way to get the project canceled. Don’t present the budget numbers with a negative attitude. Instead, present a can-do “we can raise the money!” attitude about the massive costs, wait a week or two, and it’ll be quietly moved to the back burner in the next big emergency.

PROTIP: If the big budget doesn’t scare the top brass, then celebrate! Your company is awash in cash and you will get loads of experience on some sweet new equipment. Don’t worry about the waste. Seriously, if you can get it all to work, you’ll have some of the funnest days at your job, ever. If not, well, keep up appearances and it’ll still probably get moved to that back burner in the next big emergency.

Networking Lab of Doom

Sunday, August 25th, 2013

If you’re just now learning about networking, try this one out for fun.

1. Get a switch. Plug in a few client PCs and devices. Give them all static IP addresses on the same subnet.
2. Have the devices ping each other, maybe access a file share or two. Life is good, right?

Well, that’s not the lab. It really starts in step 3.

3. Take a network cable and plug both ends of it into the switch.
4. NOW try to ping or share files. Not so good, is it?

Watch what the switch does between plugging in the cable and unplugging it. Learn to spot the difference because if you see this thing in real life, you will want to be able to stop it, and quick.