Neat, nice work! Doing this with Selenium and a (headless or not) browser is really excessive though. You should be able to do it with Requests pretty easily. I'm not sure if there's some extra...
Neat, nice work!
Doing this with Selenium and a (headless or not) browser is really excessive though. You should be able to do it with Requests pretty easily. I'm not sure if there's some extra complexity that's not coming to mind, but I think it should just involve:
Send a GET request to https://tildes.net/login and get the CSRF token value out of the HTML (it's in both a <meta> tag and a hidden input in the form).
Send a POST request to https://tildes.net/login with the login info and that CSRF token (as csrf_token).
Send another POST request to https://tildes.net/~whatever_group/topics with that cookie data, the same CSRF token and all the data you want to use for the new topic (title, link/markdown, tags)
If you're new to it, http request can seem pretty complicated. However there are a ton of libraries available that simplify the standard kinds of requests, handling post data, cookies, etc. Every...
If you're new to it, http request can seem pretty complicated. However there are a ton of libraries available that simplify the standard kinds of requests, handling post data, cookies, etc. Every language has several, and at least one favorite one. Axios is my preferred JS one.
That said, working through the details of how an http request is assembled, sent, and resolved is a good learning exercise. The protocol is pretty straight forward, especially for basic things like "go get this html".
Using requests, I had to take some extra steps to post something (though it was pretty easy to figure out going from your description): import requests; from lxml import html; login_page =...
Using requests, I had to take some extra steps to post something (though it was pretty easy to figure out going from your description):
importrequests;fromlxmlimporthtml;login_page=requests.get('https://tildes.net/login');csrf_token=html.fromstring(login_page.content).cssselect('meta[name=csrftoken]')[0].attrib['content'];cookies=login_page.cookies;headers={'referer':'https://tildes.net'};login_request=requests.post('https://tildes.net/login',data={'csrf_token':csrf_token,'username':'-snip-','password':'-snip-'},cookies=cookies,headers=headers);new_topic=requests.post('https://tildes.net/~test/topics',data={'csrf_token':csrf_token,'title':'title','markdown':'a b c','tags':'a,b,c'},cookies=cookies,headers=headers);
What are the most significant risks of a CSRF attack for Tildes? Most likely stolen account credentials, possibly posting/editing in spam of some kind, or just general trolling (e.g. deleting all...
What are the most significant risks of a CSRF attack for Tildes? Most likely stolen account credentials, possibly posting/editing in spam of some kind, or just general trolling (e.g. deleting all of the user's posts). Stealing account credentials would just be simple phishing and wouldn't require a CSRF attack (unless the intention is to act as a proxy and remain undetected), so we can probably strike that out. The big question, then, is how risky do we consider the insertion of spam or automation of trolling behaviors like destroying post history? Since this sort of attack could theoretically be triggered by simply visiting a malicious website while logged in to Tildes, I would say it's a fairly significant risk. Probably not likely, but certainly not something we would want to disregard.
The good news is that, to my knowledge, the Referer header is considered "forbidden" and thus not possible to modify programmatically in the browser (at least not in any browser using modern security standards). With that in mind, I have a recommendation: if possible, whitelist the cases where the header is provided and matches the expected and where the header is not provided/is empty.
This would provide maximum flexibility while limiting the risks to phishing (already a possibility), unofficial app installs (also already a possibility), and opting out of browser security features, all of which are squarely the fault of the user and not of lax security on the part of Tildes.
I'm not sure how feasible/workable this is with Tildes' architecture as I haven't had the time to really dig into it, but it may be an option worth exploring.
As a separate question, I know you were testing on the live site in ~test because you make suspicious spikes on my graphs like this (okay and also you told me you were doing it), but have you...
It should work exactly the same for testing scripts like this except that you point them to https://localhost:4443 instead of https://tildes.net, and that way you could probably be a little less worried about doing something harmful to the real site.
Edit: actually, it might need a little more than that because some libraries might not be happy about the self-signed SSL certificate that get used on the development version, but it's usually just an argument or something to say that you don't care about the certificate being valid.
This strikes me as something that really ought to be built right in to Tildes someday, along with a simple calendar to help manage it. Good future project when it gets to the point where lots of...
This strikes me as something that really ought to be built right in to Tildes someday, along with a simple calendar to help manage it. Good future project when it gets to the point where lots of groups are busy enough to have their own teams moderating and mod tools really get started. Nice work. :D
https://tildes.net/~tildes/dyj/scheduling_posts I totally forgot to add a gitlab issue when @Lionirdeadman first submitted that though.... but I have done so now:...
Looks like the password and username live in the config.py file, based on the GitLab repo. As long as you kept your cloned copy of the repository local and didn't push it back to GitLab, the...
Looks like the password and username live in the config.py file, based on the GitLab repo. As long as you kept your cloned copy of the repository local and didn't push it back to GitLab, the plaintext password would only be accessible to you on your local machine (no one could just grab it off of GitLab or anything).
Neat, nice work!
Doing this with Selenium and a (headless or not) browser is really excessive though. You should be able to do it with Requests pretty easily. I'm not sure if there's some extra complexity that's not coming to mind, but I think it should just involve:
https://tildes.net/loginand get the CSRF token value out of the HTML (it's in both a<meta>tag and a hidden input in the form).https://tildes.net/loginwith the login info and that CSRF token (ascsrf_token).https://tildes.net/~whatever_group/topicswith that cookie data, the same CSRF token and all the data you want to use for the new topic (title, link/markdown, tags)That should be it, I think.
If you're new to it, http request can seem pretty complicated. However there are a ton of libraries available that simplify the standard kinds of requests, handling post data, cookies, etc. Every language has several, and at least one favorite one. Axios is my preferred JS one.
That said, working through the details of how an http request is assembled, sent, and resolved is a good learning exercise. The protocol is pretty straight forward, especially for basic things like "go get this html".
Using requests, I had to take some extra steps to post something (though it was pretty easy to figure out going from your description):
CC @hungariantoast :D
Ah, right. The
Refererheader would be necessary too because I have an origin check to help try to prevent CSRF too: https://github.com/OWASP/CheatSheetSeries/blob/master/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.md#user-content-verifying-origin-with-standard-headersI may want to drop that check though, since it seems like it causes issues for people that have disabled sending referrer info: https://gitlab.com/tildes/tildes/issues/128
What are the most significant risks of a CSRF attack for Tildes? Most likely stolen account credentials, possibly posting/editing in spam of some kind, or just general trolling (e.g. deleting all of the user's posts). Stealing account credentials would just be simple phishing and wouldn't require a CSRF attack (unless the intention is to act as a proxy and remain undetected), so we can probably strike that out. The big question, then, is how risky do we consider the insertion of spam or automation of trolling behaviors like destroying post history? Since this sort of attack could theoretically be triggered by simply visiting a malicious website while logged in to Tildes, I would say it's a fairly significant risk. Probably not likely, but certainly not something we would want to disregard.
The good news is that, to my knowledge, the Referer header is considered "forbidden" and thus not possible to modify programmatically in the browser (at least not in any browser using modern security standards). With that in mind, I have a recommendation: if possible, whitelist the cases where the header is provided and matches the expected and where the header is not provided/is empty.
This would provide maximum flexibility while limiting the risks to phishing (already a possibility), unofficial app installs (also already a possibility), and opting out of browser security features, all of which are squarely the fault of the user and not of lax security on the part of Tildes.
I'm not sure how feasible/workable this is with Tildes' architecture as I haven't had the time to really dig into it, but it may be an option worth exploring.
As a separate question, I know you were testing on the live site in ~test because you make suspicious spikes on my graphs like this (okay and also you told me you were doing it), but have you tried setting up a dev version locally with Vagrant?
It should work exactly the same for testing scripts like this except that you point them to
https://localhost:4443instead ofhttps://tildes.net, and that way you could probably be a little less worried about doing something harmful to the real site.Edit: actually, it might need a little more than that because some libraries might not be happy about the self-signed SSL certificate that get used on the development version, but it's usually just an argument or something to say that you don't care about the certificate being valid.
This strikes me as something that really ought to be built right in to Tildes someday, along with a simple calendar to help manage it. Good future project when it gets to the point where lots of groups are busy enough to have their own teams moderating and mod tools really get started. Nice work. :D
https://tildes.net/~tildes/dyj/scheduling_posts
I totally forgot to add a gitlab issue when @Lionirdeadman first submitted that though.... but I have done so now:
https://gitlab.com/tildes/tildes/issues/497
Where is the user's password stored, and who has access to it?
Looks like the password and username live in the config.py file, based on the GitLab repo. As long as you kept your cloned copy of the repository local and didn't push it back to GitLab, the plaintext password would only be accessible to you on your local machine (no one could just grab it off of GitLab or anything).
config.py should ideally be added to the repo's .gitignore file so that nobody risks making this mistake.