When I input the URL in facebook debugger(https://developers.facebook.com/tools/debug), I get the following messages:

Scrape Information
Response Code 404
Critical Errors That Must Be Fixed
Bad Response Code URL returned a bad HTTP response code.
Errors that must be fixed
Missing Required Property The 'og:url' property is required, but not present.
Missing Required Property The 'og:type' property is required, but not present.
Missing Required Property The 'og:title' property is required, but not present.
Open Graph Warnings That Should Be Fixed
Inferred Property The 'og:url' property should be explicitly provided, even if a value can be inferred from other tags.
Inferred Property The 'og:title' property should be explicitly provided, even if a value can be inferred from other tags.

Why is facebook not reading the meta tags info? The page is accessible and not hidden behind login etc.

UPDATE

Ok I did bit of debugging and this is what I found. I have htaccess rule set in my directory- I am using PHP Codeigniter framework and have htaccess rule to remove index.php from the url.

So, when I feed the url to facebook debugger(https://developers.facebook.com/tools/debug) without index.php, facebook shows a 404, but when I feed url with index.php, it is able to parse my page.

Now how do I make facebook scrape content when the url doesn't have index.php?

This is my htaccess rule:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
#Removes access to the system folder by users.
#Additionally this will allow you to create a System.php controller,
#previously this would not have been possible.
#'system' can be replaced if you have renamed your system folder.
RewriteCond %{REQUEST_URI} ^system.*
RewriteRule ^(.*)$ /index.php?/$1 [L]
#When your application folder isn't in the system folder
#This snippet prevents user access to the application folder
#Submitted by: Fabdrol
#Rename 'application' to your applications folder name.
RewriteCond %{REQUEST_URI} ^application.*
RewriteRule ^(.*)$ /index.php?/$1 [L]
#Checks to see if the user is attempting to access a valid file,
#such as an image or css document, if this isn't true it sends the
#request to index.php
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php?/$1 [L]
</IfModule>
<IfModule !mod_rewrite.c>
# If we don't have mod_rewrite installed, all 404's
# can be sent to index.php, and everything works as normal.
# Submitted by: ElliotHaughin
ErrorDocument 404 /index.php
</IfModule>

If you have an .htaccess file redirecting things and making it difficult for Facebook to scrape your URL you might be able to get away with detecting Facebook's crawler with your .htaccess and feeding it the correct tags. I believe the the user agent that the Facebook crawler provides is this :

The [L,QSA] flags that I placed there state that this is the L​ast rule that will be enforced on the current request (L) and the QSA (Query String Append) states that any query string given will be passed along when the URL is rewritten. For example, a URL such as :

https://example.com/?id=foo&action=bar

Will be passed to ogtags.php like this - ogtags.php?id=foo&action=bar. Your ogtags.php file will gave to generate dynamic og:meta tags according to the parameters that were passed.

Now whenever your .htaccess file detects the Facebook user agent, it will pass him the ogtags.php file (that can contain the correct og:meta information). Please be aware of any other rules you have in your .htaccess and how they might affect new rules.

From the .htaccess entries that you have detailed, I would recommend placing this new "Facebook rule" as the very first rule.

Hi Lix, thanks a lot for the update. I have an issue though- in the rewrite rule, you have mentioned that I load ogtags.html but the meta tags will have dynamic content, based on the page that is being requested. I can't give a static html page there. I tried replacing ogtags.html with this rule: RewriteRule ^(.*)$ index.php?/$1 [L] but didn't help. Any thoughts on how to achieve this?
–
NinjaApr 11 '12 at 7:46

@Lix: do you have any idea why I get a 500 error from the facebook debugger tool when I use your two rules? thanks in advance...
–
sergioJul 29 '13 at 18:33

Hey there @ser - Have you checked your server logs for requests from Facebook that are being denied? I just added this link to my answer here, it might be useful to you too.
–
LixJul 29 '13 at 18:40

@Lix: thank you very much for your reply! strange thing is: facebook debug tool can access mysite.dom/ogtags.php but for mysite.dom it returns 500... from the server logs I get 206 for mysite.dom/ogtags.php and 500 for all URIs within ogtags.php (e.g., og:image)... I see now that there could be an infinite recursion going on...
–
sergioJul 29 '13 at 19:26

I had the same problem, which was:
Bad Response Code: URL returned a bad HTTP response code.

but oddly this is what solved it:
I've added

<meta property="og:locale" content="en_US" />

to my site HEAD tag and it worked.

Also, not to forget, in your application dashboard (where you get your APP ID) you must have atleast "Website with Facebook Login" enabled and enter the URL of the website.
Otherwise it won't work...regardless if you are not using any Facebook Logins on your site.