9 replies to this topic

[Chapman]

I would like to set up a WebDAV area on our server so that I can sync programs (mobile devices and computers) and post files. I have gone to cPanel - WebDisk. Once I created an account and password I got a message that the directory would be in "public_html/…" There was a warning that these files would be publicly available. I would like it so that they are not publicly available and that google, robots, or whatever is out there would not be able to find or access them. How do I do this?

[-ASO- Aimée]

This should be helpful:

http://www.antezeta....engine-indexing

[IBBoard]

That only works for bots that pay attention to robots.txt and the like, though (except for password protecting, and I'm not sure how that works with WebDav).

I've just had a quick poke, and as I see it then you have two options:

1) Put the WebDav folder outside of public_html (you can do it, it is just that cPanel is security illiterate enough to default to putting it there).
2) Password protect the directory using the cPanel "Password Protect Directories" options. The message should contain a link to point you straight at it.

[-ASO- Aimée]

IBBoard, on 02 April 2011 - 10:55 AM, said:

That only works for bots that pay attention to robots.txt and the like, though (except for password protecting, and I'm not sure how that works with WebDav).

The link has several options outside of robots.txt, including password protection.

[IBBoard]

Yeah, I know, I gave it a quick skim. Apart from password protect then they all seemed to be robots.txt or its equivalents (meta tags, etc).

At the end of the day, the only way to be sure it isn't accessible on your website is to not put it in a folder under your website (or password protect it). Other than that then it is pretty much always going to be accessible to someone who find it.

[Chapman]

Yeah, I use the robot.txt already. I was hoping for something more reliable. I'll look into the rest of the options.

[NyteOwl]

Actually, malicious robots can use robots.txt to do searches in directories they might not otherwise know about.

[-ASO- Aimée]

NyteOwl, on 02 April 2011 - 03:06 PM, said:

Actually, malicious robots can use robots.txt to do searches in directories they might not otherwise know about.

Smart! Sneaky, but smart.

Edited by Aimée, 02 April 2011 - 05:18 PM.

[IBBoard]

Same as real life, I guess. Where's the best place to look for secrets? The unlocked door that says "please do not enter - authorised personnel only". That's about all that robots.txt and the rest are - a little sign saying "would you mind awfully not going here, or not caching/listing this page?"

Edited by IBBoard, 03 April 2011 - 07:57 AM.

[NyteOwl]

Yep. Same goes for the directives placed in hhtp headers on in meta statements. Following them is entirly voluntary on the part of the 'bots'.

That's why I have a bot trap on my site for the ones that don't obey directives