The Dangers of Memcache Vulnerabilities and NPM Module Exploits

Memcache, a popular caching system, has been found to have a vulnerability that can lead to serious exploits. When an instance variable is serialized, it can contain a command that is executed on the server when deserialized. This means that if an attacker can manipulate the data being stored in memcache, they can potentially execute arbitrary commands on the server.

A recent example of this exploit was found in a memcache key that contained a piece of code. The key was used to store and retrieve data from the memcache system, but it also contained a command that would be executed when the data was deserialized. This means that if an attacker could manipulate the data being stored in memcache, they could potentially execute arbitrary commands on the server.

The vulnerability was exploited by someone who had created their own memcache module called "cross" without the hyphen. The real cross module, which is widely used, has a known vulnerability in its instance variable proxy, and the hacker took advantage of this to exploit the system. The exploitation allowed the attacker to execute commands on the server, potentially leading to serious security issues.

A similar example was found with an NPM module called "cross" that contained environment variables as input parameters, but also included a piece of code that would be executed when the data was deserialized. This means that if an attacker could manipulate the data being stored in the package manager, they could potentially execute arbitrary commands on the server.

The moral of the story is that big exploits are made from smaller exploits. A security vulnerability that seems insignificant at first can be used to launch a larger attack. This highlights the importance of identifying and fixing vulnerabilities quickly, rather than relying on the assumption that the inputs will come in the expected format.

In addition to these examples, there is another NPM module called "cross-ends" which has been found to have a vulnerability. The module takes environment variables as input parameters, but also includes code that would be executed when the data was deserialized. This means that if an attacker could manipulate the data being stored in the package manager, they could potentially execute arbitrary commands on the server.

The lesson from these examples is to always sanitize and validate user inputs, especially when dealing with environment variables or other types of input parameters. It's also important to remember that just because a vulnerability has been fixed, it doesn't mean that all instances of the affected module are secure. The only way to be sure is to keep your code up-to-date and monitor for any signs of exploitation.

In addition to these security lessons, there is an NPM module called "cross" that can be used to create a custom platform to run software on. This means that developers can take control of their own platforms and ensure that only trusted inputs are executed. This highlights the importance of being mindful of dependencies and ensuring that all inputs are validated before execution.

The Dangers of Environment Variables

Another example of how environment variables can be exploited is with a setup script for an NPM module called "cross-ends". The script takes environment variables as input parameters, but also includes code that would be executed when the data was deserialized. This means that if an attacker could manipulate the data being stored in the package manager, they could potentially execute arbitrary commands on the server.

The script is designed to take a string of random variables and post them to a server. However, it's possible for an attacker to manipulate the input parameters to send more than just random variables to the server. This highlights the importance of validating user inputs and ensuring that only trusted data is sent to a server.

The moral of this story is to be careful when using environment variables in your code. It's not enough to simply trust that the inputs will come in the expected format. Instead, you need to validate and sanitize all user inputs to prevent exploitation.

Conclusion

In conclusion, these examples highlight the importance of identifying and fixing vulnerabilities quickly. A security vulnerability that seems insignificant at first can be used to launch a larger attack. This emphasizes the need for developers to always prioritize security and take steps to protect their code from exploitation.

By following best practices such as sanitizing user inputs, validating dependencies, and being mindful of environment variables, developers can help prevent exploits like these in the future. It's also important to stay up-to-date with the latest security patches and updates for your dependencies to ensure that you're protected from known vulnerabilities.

In addition to these technical measures, it's also important to be aware of the potential risks associated with open-source software. While open-source software can provide many benefits, it can also introduce new risks if not used carefully.

By being informed and taking steps to protect yourself, developers can help prevent exploits like those described in this article. And by prioritizing security, developers can help build more secure software systems that are less vulnerable to exploitation.

References

* GitHub issue report for NPM module "cross-ends"

* GitHub issue report for NPM module "cross"

* No Jet Security course on memcache vulnerabilities

* No Jet Security course on NPM module exploits

"WEBVTTKind: captionsLanguage: enall right every hey me cool so before I begin I just like to ask a really quick question could you put your hands up if you've ever been so scared in your life so frightened that a little bit of pee has come out keep your hands up okay that's about 50% of the audience and there hopefully by the end of this talk I will have increased that number a little bit because today we're talking about web security and hacking my name is asked to me same on Twitter you can find me at jaw 8 I blog about JavaScript and angular on my site code Croft TV and I'm something called a cloud developer advocate and Microsoft so I work on the is your team or with the is your team and yeah I talk about cloud and security net and I and JavaScript in a bunch of those things but to begin with today I want to talk to you one tell you a story the story of my first startup it's called event sushi it was an event site that aggregated events from a bunch of different places from Meetup Facebook last.fm a few of the places at the time I used to work in investment banking my career was going very well there was good money but this just this was just a side project but you know it was going slightly ok very very slightly so I did where everybody does in this situation on that I quit a really good job well paying to work on something quite risky and I told you I was in investment banking so of course I was very very arrogant I had it at the start had investors who are interested investing in me but I was like no no no no this is gonna be a billion dollar startup I don't want your money I want a hundred percent so I said no so you can probably guess where I'm going with this six months later the product hadn't you know didn't wasn't a billion dollar startup I was running out money this is my money not invest as money my money I didn't have enough money to pay rent in two months so I didn't start going after the same investors saying hey look I'm interested now but they could tell they could tell that I was now desperate they were backing away they didn't want to invest a pot from one guy we've been talking for six months he lied me you liked the product and he said asked him okay look give me another demo in seven days show me what you built recently and then we'll talk and it was a good chance who's gonna invest yes but then 48 hours before the meeting with the investor I got emailed by Leonard Cellino were the people I was using to host my website and they said look assam it looks like your servers being compromised we can see it's making hundreds of requests to other servers and if you don't solve the problem in four in 24 hours we're gonna shut you down so in 48 hours I was gonna have a life-saving meeting with an investor and in 24 hours I was gonna get shut down by Linode so I panicked you know jumped onto the server I started looking around and to cut a long story short I definitely got hacked definitely got hacked I found stuff in a temp folder there were scripts that were running that was doing brute-force attacks on a bunch of IP addresses I'm not talking like what what is going on how did I get hacked and then after a bit more investigation I saw the PHP was running why is PHP running I'm not a PHP developer right then I remembered when I first set up the server I'd installed WordPress okay I got rid of it straight away but I forgot to remove it from the start script so when the server rebooted PHP started and that's how the hackers got in so you know I cleaned everything up removed from the start script contacted they know they were happy had the meeting with the investor he invested the startup still failed but had some money so I'm no fool I'm not an idiot I took security very very seriously when I set up that server I'd followed all of the instructions I just forgot on one tiny tiny tiny small thing so I think the lesson is that if it can happen to me it can happen to you so you might guess that I love stories so today I'm gonna talk to you about hacking but through a series of four different hacking stories some of these stories are going to be specific to notes um some are gonna be more general web development each story's gonna have a moral at the end a lesson to learn and perhaps some steps you can use to protect yourself and I think to begin with let's just start off by breaking down exactly how I think I got hacked on event sushi but first let me just explain a few terms so a vulnerability is a hole in your security a weakness so for instance not setting up a firewall is a hole in your Security's of vulnerability an exploit is a tool or a piece of code or even just a sequence of your commands that you execute which takes advantage of a vulnerability to do bad things okay so who here has heard the term zero-day exploit yeah so a zero-day exploit is one that nobody knows about yet okay crew but once a zero-day exploit is found out about is known about perhaps a white hat hacker has discovered it and informed the company it's not called a zero-day exploit anymore anybody call their a1 there or 3:30 or a six-month expletive yeah will you call it that but just to think about just the way to think about it basically once a zero-day exploit becomes known the clock starts ticking let me ask you a question how hard do you think you would be to get a hold of a zero-day exploit pretty hard right now you need to be the right kind of person you need to know the right kinds of people there's a study done caramba when maybe last year they estimated that a zero-day exploit is sold for about quarter million dollars each the usually salt to criminal organizations or state-run hacking organizations but how hard do you think it would be to get a hold of a six month old exploit not very hard in fact it's it's just it's actually really really easy you can just find on the Internet this is one website exploit DB listless bunch of sizes this is just one of them and it's pretty easy to use siphon PHP it's got a capture hit search and take a look at this these are all the exploits is found look at the date we're still in 2017 PHP isn't very secure okay I can tell it got some PHP developers in the audience so we want you know this yeah mr. robot so we all like to believe all of the Oliver hackers are like this blackhat mysterious geniuses and I think we like to believe that because it makes us feel better when we do get hacked I mean if we got hacked and you can just say well how am I supposed to protect myself and mr. robot come on but mr. robot didn't hack me didn't use a zero they exploit to get into my server I was running a really old version of PHP my attacker googled how to hack me and followed instructions it's not hard in fact it's even easier than that I mean we're all developers in this room how many of us has been a day writing a script to automate something that takes us two minutes we all do all the time so why would hackers do the same and they do all the time this is a automated tool called Metasploit which you can basically point to a website and it will scan it for known runner built vulnerabilities and it also has a bunch of plugins in here and you can store other plugins that you want that will also perform automated attacks as well there's another study that estimated about 27% of all websites in the world can be hacked automatically so what can we do what can we do to to protect ourselves from this let's not not much you can do about zero day exploits okay you still you can't protect yourself from something you don't know about but you can defend yourself from the unknown all you've got to do is make sure you keep all of your stuff we're updated all the time you know make sure you keep the webserver updated oh the operating system itself make sure you keep that updated oh I'm using nginx gotta make sure I'm keeping tap dated and on my database gotta make sure I'm gonna keep my database updated all the time and oh my god I'm using em PMS need to make sure man p.m. modules of data real-time also need to listen to security bulletins as well and make sure I patch my software as soon as soon as anything gets updated especially not that easy and your job isn't to do all of that stuff your job is to write functionalities to write software for your end-users and that's why after I got hacked this one time I've started to exclusively use houses just explain what house is back in the old old days or even now if you sold it now and if you want to deploy something yet to have some sub on-premise Hardware you'd buy a server you maybe you hope you put it in your own room or you go to server farm and they will just put it in there for you but you're in charge of the hardware if the hard drive fails is up to you to fix it but then you're also responsible for the operating system keeping that updated and the web server run the database and everything else and your application as well you've got obviously keep updated then we have infrastructure as a service so that stuff like what Linode Amazon Web Services is your Google we've all got stuff like this and that's basically when you get a VM so somebody else is responsible for looking after your hardware if the hard drive fails they fix it but you still have to handle the operating system the web server with a database everything else and of course your application as well and then we have passes platform as a service so they handle the hardware they also handle the operating system depending on the powers of another web server and a database and everything else as well and you just focus on deploying your application and the thing with passes is they've got teams of people teams or security experts and all their job is every single day is making sure there's no security holes patching every vulnerability as soon as it gets released okay I know there's a bunch of them out there I work in Microsoft so we've got one called as your app services Google's got Google App Engine Amazon Beanstalk and there's another one I've used Polaroid but there's a bunch of others as well so and if you're still kind of a little bit and there's a lot of people who are kind of used to using VMs and installing all around software and managing all this stuff themselves that a little bit suspicious of what I'm saying who's heard of the Equifax hack happened earlier on this year ok it's the largest ever hack in history but 200 million people's records were taken Equifax the billion dollar company about ten thousand employees did the hack is gained through a zero-day exploit no they got in through a known exploit of Apache struts which is a Java web framework patch to the fix was available for two months before the hackers got in all the Equifax had to do to defend from this attack was apply an update that just never updated their software largest hack in history Oh in the firm that cook one of the coolest things about is your that I like is something called he is your security center so as far as I can tell and correct me from wrong no one else has got this yet so one of the problems with detecting attacks is as low as a different signals all over the place and if you just alerted I'm every single signal then that just be too much noise and you start ignoring the ellipse so to solve this we trained in AI so we took a whole load of attack data we said this hall at a date so we said these signals are from definite attacks we trained up an AI so now we've beaten all of your signals through this ai and we alert you only when well the AI alerts now it's not perfect but it's easy just switch a button it switched on and I'm lazy so I just have to switch it on does anybody know this show nobody don't worry about it I think I'm one of the three fans of this show in the world it's called this so it's always sunny in philadelphia very funny I recommend a finger on the eleventh season now and this is one of the characters called Charlie Kelly and he plays the dumbest guy in the show he like cleans the toilets and kills rats that make sense as you watch it but one of the things is he thinks he thinks he's a lawyer so never he gets into trouble he always represents himself in court very badly represents him himself in court so my closing arguments of this thinking you can create a secure platform to host your application when you're not a security expert is like thinking you can represent yourself in court when you're not a lawyer so did I scare anybody just yet no you're all pretty pretty calm I can tell there's no I can't we see the fear in the you I can't smell it maybe the next one so I'm in the UK I'm based in the UK in the UK when you register a company you have to register with the government was something called Companies House so if you actually went to Companies House website and search for my company Dell Reaver Limited you would find me here and this is an actual company in the UK space this is the address yeah and what this is this is an example of an attack something called an injection attack something specifically called a sequel injection attack and the point of kind of any injection attack is to run untrusted code in a trusted environment so basically to trick you in running this sequel code here so how does that work maybe you've got a script which gets all the names and you want to grab some details from your database but if you actually put the name of the company in here you'd end up with something like this select star for a company where name is you mmm nothing drop table companies and then a comment man you might be saying so what a Seng I've backed up my company's table I'm not really bothered about this I'll just restore it but then the drop table isn't the only command you can run on a database and just like a lot of things there's automated tools which help to attack so look at this on sequel map automated SQL injection and database take over to take over so what does this do well you would run it there we go it's a Python script pass it some endpoint there's an API endpoint which has a injection attack vulnerability if they're scans it figures our is my sequel the version number different aspects about the database and then okay now we know it's my sequel let's try it with a common set of passwords for my sequels it's brute-force attacking and there we go found the password for it now let's have a list of the databases let's list all the tables that's now what's it gonna do now oh yeah let's dump a table let's dump the users table yeah okay no biggie but my sequel also lets you do things like OS shell which they should run commands on the server my sequels running on so we're running LS and now we're catching the password file all from one vulnerability in your script so in about a minute that's a minute in about a minute we ran that script and got the egg set to pass a file from your server awesome morale this story acting the moral of this story is never assume that your inputs will arrive in the format that you expect okay and the kind of solution for this and this kind generous across a lot of problems a lot of whole vulnerabilities in this base is to do something called sanitization okay you're you've got untrusted input that name was untrusted someone else is providing that so you have to sanitize it it's a bunch of different methods of sanitizer you want notice equal you can use sequel string where you pass it some untrusted sequel it will escape stuff that it knows is dangerous and that you run the output command output sequel and if you're using as your sequel database we also have detection capabilities or you can detect injecting sequel statements that we know are suspecting are basically injection attacks and we alert you on there we don't stop the sequel from being execute we just alert you on it and then feed him it's their AI as well so you can change you can actually see in a chain how we think you got sequel injection attacks oh we think that somebody logged into your server and you can follow this path through your system so that was a second story I'm feeling now a little bit more why I did I did just show everybody how to completely take over a server in a minute using okay you get a tough crowd well I don't know maybe the next story anybody heard of this this company no I still use them perforce so this is github we all know github hopefully and github has something called a bug bounty where basically they all like pay you if you find a security hole in their software and you tell them about it first you tell them about it privately and you give them a chance to fix it before you tell everybody in the world so there's a really great exploit found by this user called orange sigh this is their Twitter follow them I think posting some of the interesting stuff and it was with github Enterprise and they basically found a way to run any command on a github server the server itself okay and they did this by chaining a number of smaller exploits together into one really big exploit and and I really liked it because it reminded me of like a heist movie know movies where they rob a bank this story really reminds me of something like that so I'd love to share it with you so we know webhooks or with maybe we know web hooks and github where you can set up github so when you do a git push it does a post to some end point that you define pretty straightforward right but what if you set up the web Booker's blog post mm-hmm what what happened then well then when you do a git push behind the firewall it would then do a post to something internal to the server remember you can have localhost with a port number so you can post to any process running on that same machine but you know this is github very known Beneful x' they were using a sanitizer because that is untrusted input and that sanitizer knew the local host was a potential problems always strip it out but forgot about zero okay on some server zero would also point to localhost so then okay found a way in so now we've got one small vulnerability where you can do a git push and post to some process on your server but what what can you actually do with that well the server was also running elasticsearch we know elasticsearch that runs on port 9200 and if you do a post trip hate HTTP POST request to this port and this endpoint it shuts down elasticsearch but you know it's not a biggie it's not a big no biggest issue in the world then started looking for another vulnerability to chain onto this start looking at the code and found this okay so I know this is - don't worry about it I'll take you through so the server is also running another process called graphite was the Python process that lets you do kind charting and things like that and it's open source so it's checked into github so on github you found the source code for it and then this was one of the functions as it gets called on a on an API request and just to take you through what this is doing it gets there the URL from the query parameters gets the path and then does it get request to that path does it get request that path so what we're talking about here well now if you set your your web hook to this URL 800 is graphite when you do a git push it will post to this endpoint that endpoint is that send email function it will take this URL and then do a get request so all that these done is have turned a post into a get a post into again on its own maybe not such a big deal but but the actual method their actual library they use them to make that request it's called HTTP connection it's a Python library that has a known vulnerability something called carriage return line feed so Kara turn the backslash R is how you did new lines and windows and backslash n is new line everywhere else so but you know if you convert that to hex it would look like this oh do a so now what happens when you make a get request to this okay so when converted into the protocol they might look like this but hey CP connection library will convert these two new lines and then we're sending this okay so what is HTTP what is it what is hasty P HTTP is you open up a TCP connection to this host and port and then you're sending it a bunch of strings and if that host and port recognizes those strings it goes oh this is HTTP connection with a snippy message I know what to do but if it gets something this doesn't look like HTTP it's just going on this is a malformed the message it's going to kick it out of the way right that's what's happening here okay so what you can send a malformed haste VP message to the server but what if you did this okay that port is memcache now if you try to do a get request to this you actually end up sending a HTTP method message like this again we're opening up a TCP port to memcache memcache doesn't know what this means oh it's back memcache doesn't know what the first line means this is gonna ignore it right the second line is a memcache understands this it knows what this is this is setting the key the value data into the key key so it will do something with that it will then ignore this line at this then this thing doesn't know what these are okay what have we done we've done something called protocol smuggling okay in a HTTP message we've managed to smuggle the memcache protocol and so now we can set data in memcache oh I forgot okay so as developers we all like to store stuff in memcache and restore some Co serialize it store in memcache deserialize it execute it we're happy but we're all pretty lazy people so we probably don't do that serialization manually you probably use a library where you pass an object serializes it for us thaws in memcache take it back the lab your uncie realizes it I'm the executor sometime but now we can set some data in memcache so as a developer you're going to serialize your objects you're going to store it in memcache later on you're gonna deserialize your objects but you're not deserializing your object but deserializing my object you're gonna execute a function not get to execute your function you can execute my function okay I'm looking through the keys in the memcache found this so when you say realize a bunch of objects you might depending the library might store the name of the class which is getting serialize so they were using deprecated instance variable proxy instance variable proxy had a known vulnerability so they put deprecated in front of it but they still used it so then it basically had a vulnerability where if you deserialize an instance of instance variable proxy you can execute a command so then this is what the eventual webpop looks like you've got the first one it's that simple posts gets converted into a the second one to get then this is the memcache key so you can do set that's the key this is the deserialize data and icon where it is somewhere in here this is it this is the command when we deserialize this object the ID command gets run on the server so this is what they used to prove their case and get get the money examples are working you go to your profile click on a repository settings add a web for runs a script to get the eventual name okay gets their webhook URL puts it in gets ready to listen to any command to get run on the server and then to run that thing just basically does a search any search that's going to request that data from memcache the D serialization then runs the ID command on the server hmm the moral of the story big exploits are made from smaller exploits okay we like to think are our attacks coming through one big giant security hole but they don't you chain multiple smaller exploits together so you know if you find a vulnerability in your software anything yeah it's not that big a deal now fix it cuz the github is vulnerability the only thing they had in their code pretty small how you feeling now a little bit more okay I've got a rush to the next one but we'll go through fast all right what does this code do it takes your environment variables see that there's guess the string of um random variables okay what does this code do what's why is my surprise my server there Oh if you ran this code it looks like it would take all of your environment variables and post them to my server what if I told you I could make you run this code on your server who here puts like private things in there and random variables I don't yeah okay maybe maybe this makes it clearer what this is I should get an extra minute for this maybe this makes it clearer it's some sort of setup script in fact maybe this makes it clearer it's actually a an NPM module and this is a setup script for an NPM module if you install this NPM module you would send me all of your environment variables okay he probably saying Assam iodine store your npm module he saw this is Kirk almost two months ago now so Kent Dodds had a npm module called cross ends and they found that somebody else had create their own cross named module the real one was crossed hi for them the fate one was cross and without the - this is downloaded a million times a day by the way so if you installed cross them without the hyphen you are sending your environment variables to somebody - hacker how many times have you sat then and can't quite remember the npm module names you try it without the hyphen any like yeah would great that's what happens so the moral the stories were a little bit too trusting okay and I think maybe it's because it was open source we kind of trust open source data we don't really question it now MPM have taken down cross them they've they've got rid of all the ones they could find associated with that account but you can't really protect yourself from that okay well you can do with NPM now as you can absolutely call scope packages basically you can own the scope and then only you can publish things under it that's a solution to you stopping yourself getting typos spotted on which is what that's called but it doesn't stop you from accidentally installing something like cross them yeah just needs to keep your your mind clear on that one just in summary okay what the takeaways stop pretending stop retain that because you spend a few minutes thinking about security that you can create your own platform to run software and use the pass don't assume the inputs will come in the format that you expect sanitize don't think that just there's just a small one durability acting Nora fix it don't trust anyone and I don't have a solution for that so there's a unicorn and and that's it if you do want to know a little bit more my colleague has a course on no jet security I'll post up these slides on Twitter in a second so you want to follow me follow me at joy and then now post up all the slides and I'll have links for everything and I've just mentioned in this course in this course in this talk including all the links to all the vulnerabilities and things I mentioned and that's it very much\n"