事象発生日:2017-08-02
記事公開日:-
アクセス数:3507
このブログは,Googleでの検索に引っかかるのを避けるために,
<meta NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW,NOARCHIVE">
を指定している.
一部記事のみ検索結果に表示されてもいいなと思い,Google Botによるindex保存を許可するページを,
<meta NAME="ROBOTS" CONTENT="INDEX,FOLLOW,NOARCHIVE">
許可しないページを
<meta NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW,NOARCHIVE">
とし,挙動を観察することにした.
Google Search Consoleより,index保存されたページは0である.
したがって,
[[ site:meltingrabbit.dip.jp ]]
とGoogle検索してもヒットしない.
Search Console > クロール > Fetch as Google より,
http://meltingrabbit.dip.jp/blog/article/2017042001/(【LaTeX】WindowsのSublime Text 3でのupLaTeX環境構築)
のindex登録をリクエストする.
このページは,
<meta NAME="ROBOTS" CONTENT="INDEX,FOLLOW,NOARCHIVE">
としたページのうちの1ページである.
下にApacheアクセスログを示す.
UAが
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/41.0.2272.118 Safari/537.36 | |
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) の一部 |
が,おそらくFetch as Googleより取得した際のUAであり,
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) の一部 | |
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
がindex取得のためのUAだと考えられる.
index登録のリクエストの際,「この URL と直接リンクをクロールする」ではなく「この URL のみをクロールする」を選択したため,リンク先はたどってないと思われる.
# [time], HTTP Request , HTTP Referer # * [1] -> http://meltingrabbit.dip.jp/blog/article/2017042001/ UA: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) IP: 66.249.64.95 [10:22:50] "GET /robots.txt HTTP/1.1" "-" [10:23:03] "GET /blog/css/style_blog_article.css HTTP/1.1" "[1]" [10:23:04] "GET /blog/js/script_article.js HTTP/1.1" "[1]" [10:23:06] "GET /blog/img/Logo_blog.png HTTP/1.1" "[1]" [10:23:07] "GET /blog/js/script_fitting.js HTTP/1.1" "[1]" [10:23:08] "GET /blog/article/2017042001/top.jpg HTTP/1.1" "[1]" [10:24:07] "GET /etc/syntaxhighlighter_2.1.382/styles/shCore.css HTTP/1.1" "[1]" [10:24:08] "GET /css/style_default.css HTTP/1.1" "[1]" [10:24:08] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushPlain.js HTTP/1.1" "[1]" [10:24:09] "GET /blog/css/style_blog_article.css HTTP/1.1" "[1]" [10:24:10] "GET /etc/syntaxhighlighter_2.1.382/styles/shThemeDefault.css HTTP/1.1" "[1]" [10:24:11] "GET /blog/js/script_article.js HTTP/1.1" "[1]" [10:24:12] "GET /js/jquery-1.11.2.min.js HTTP/1.1" "[1]" [10:24:13] "GET /js/jquery-1.11.2.min.js HTTP/1.1" "[1]" [10:24:13] "GET /etc/syntaxhighlighter_2.1.382/scripts/shCore.js HTTP/1.1" "[1]" [10:24:14] "GET /css/style_default.css HTTP/1.1" "[1]" [10:24:14] "GET /etc/syntaxhighlighter_2.1.382/styles/shCore.css HTTP/1.1" "[1]" [10:24:15] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushPlain.js HTTP/1.1" "[1]" [10:24:15] "GET /blog/article/2017042001/style.css HTTP/1.1" "[1]" [10:24:16] "GET /css/style_default.css HTTP/1.1" "[1]" [10:24:16] "GET /blog/css/style_blog_article.css HTTP/1.1" "[1]" [10:24:17] "GET /blog/css/style_blog.css HTTP/1.1" "[1]" [10:24:17] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushPlain.js HTTP/1.1" "[1]" [10:24:18] "GET /etc/syntaxhighlighter_2.1.382/styles/shThemeDefault.css HTTP/1.1" "[1]" [10:24:18] "GET /blog/js/script_fitting.js HTTP/1.1" "[1]" [10:24:18] "GET /blog/article/2017042001/style.css HTTP/1.1" "[1]" [10:24:19] "GET /blog/js/script_article.js HTTP/1.1" "[1]" [10:24:19] "GET /js/jquery-1.11.2.min.js HTTP/1.1" "[1]" [10:24:20] "GET /blog/css/style_blog_article.css HTTP/1.1" "[1]" [10:24:20] "GET /blog/css/style_blog.css HTTP/1.1" "[1]" [10:24:21] "GET /etc/syntaxhighlighter_2.1.382/scripts/shCore.js HTTP/1.1" "[1]" [10:24:21] "GET /blog/js/script_fitting.js HTTP/1.1" "[1]" [10:24:22] "GET /etc/syntaxhighlighter_2.1.382/scripts/shCore.js HTTP/1.1" "[1]" [10:24:41] "GET /etc/syntaxhighlighter_2.1.382/styles/shThemeDefault.css HTTP/1.1" "[1]" IP: 66.249.64.66 [10:23:02] "GET /blog/article/2017042001/ HTTP/1.1" "-" [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/styles/shThemeDefault.css HTTP/1.1" "[1]" [10:23:04] "GET /etc/syntaxhighlighter_2.1.382/styles/shCore.css HTTP/1.1" "[1]" [10:23:05] "GET /js/jquery-1.11.2.min.js HTTP/1.1" "[1]" [10:23:06] "GET /blog/article/2017042001/style.css HTTP/1.1" "[1]" [10:23:07] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushJSON.js HTTP/1.1" "[1]" [10:23:08] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushPlain.js HTTP/1.1" "[1]" [10:23:08] "GET /etc/syntaxhighlighter_2.1.382/scripts/shCore.js HTTP/1.1" "[1]" [10:24:09] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushJSON.js HTTP/1.1" "[1]" [10:24:10] "GET /blog/css/style_blog.css HTTP/1.1" "[1]" [10:24:11] "GET /blog/js/script_fitting.js HTTP/1.1" "[1]" [10:24:12] "GET /blog/js/script_article.js HTTP/1.1" "[1]" [10:24:13] "GET /etc/syntaxhighlighter_2.1.382/styles/shCore.css HTTP/1.1" "[1]" IP: 66.249.64.69 [10:23:03] "GET /css/style_default.css HTTP/1.1" "[1]" [10:23:05] "GET /blog/css/style_blog.css HTTP/1.1" "[1]" [10:24:06] "GET /blog/article/2017042001/ HTTP/1.1" "-" [10:24:08] "GET /blog/article/2017042001/style.css HTTP/1.1" "[1]" UA: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) IP: 66.249.64.95 [10:24:22] "GET /blog/article/2017042001/ HTTP/1.1" "-" UA: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Search Console) Chrome/41.0.2272.118 Safari/537.36 IP: 66.249.84.227 [10:23:02] "GET /blog/article/2017042001/ HTTP/1.1" "-" [10:23:03] "GET /blog/css/style_blog_article.css HTTP/1.1" "[1]" [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/scripts/shCore.js HTTP/1.1" "[1]" [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/styles/shThemeDefault.css HTTP/1.1" "[1]" [10:23:03] "GET /blog/js/script_article.js HTTP/1.1" "[1]" [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushJSON.js HTTP/1.1" "[1]" [10:23:03] "GET /blog/article/2017042001/top.jpg HTTP/1.1" "[1]" IP: 66.249.84.229 [10:23:03] "GET /blog/css/style_blog.css HTTP/1.1" "[1]" [10:23:03] "GET /blog/js/script_fitting.js HTTP/1.1" "[1]" [10:23:03] "GET /js/jquery-1.11.2.min.js HTTP/1.1" "[1]" IP: 66.249.84.253 [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/styles/shCore.css HTTP/1.1" "[1]" [10:23:03] "GET /css/style_default.css HTTP/1.1" "[1]" [10:23:03] "GET /blog/article/2017042001/style.css HTTP/1.1" "[1]" [10:23:03] "GET /blog/img/Logo_blog.png HTTP/1.1" "[1]" [10:23:03] "GET /etc/syntaxhighlighter_2.1.382/scripts/shBrushPlain.js HTTP/1.1" "[1]"
しばらくすると,今度はUAが
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/41.0.2272.118 Safari/537.36
からアクセスがあった.
しかし,それらはindex登録の申請を行なったページではなく,HPのルートにアクセスしているため,今回の申請との関連は不明である.
# [time], HTTP Request , HTTP Referer # * [0] -> http://meltingrabbit.dip.jp/ UA: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; Google Web Preview) Chrome/41.0.2272.118 Safari/537.36 IP: 66.102.6.65 [11:41:28] "GET / HTTP/1.1" "http://www.google.com/search" [11:41:29] "GET /js/script_toggle.js HTTP/1.1" "[0]" [11:41:29] "GET /img/Logo_en.png HTTP/1.1" "[0]" [11:41:29] "GET /css/style_home.css HTTP/1.1" "[0]" [11:41:29] "GET /js/script_home.js HTTP/1.1" "[0]" [11:41:29] "GET /img/top_3.JPG HTTP/1.1" "[0]" IP: 66.102.6.67 [11:41:29] "GET /css/style_default.css HTTP/1.1" "[0]" IP: 66.102.6.95 [11:41:29] "GET /css/slide.css HTTP/1.1" "[0]" [11:41:29] "GET /css/style_toggle.css HTTP/1.1" "[0]" [11:41:29] "GET /js/jquery-3.2.0.min.js HTTP/1.1" "[0]" [11:41:29] "GET /img/top_1.JPG HTTP/1.1" "[0]" [11:41:29] "GET /img/top_2.JPG HTTP/1.1" "[0]"
Google Search Console,[[ site:meltingrabbit.dip.jp ]] のGoogle検索
のどちらにおいても,未だindex登録を確認できず.
それなりの時間がかかるようだ.
あれから特にGoogle Botが回ってくるわけでもなく,indexが登録されたわけでもなさそうだ.
まあ,このHPに対して,Google Botは月一程度しか回ってこないのでそういうものかもしれないが.
一つ心当たりがあることといえば,このHPのルートに対してindexの削除申請をしたことがあることだ.(下図)
一昔前にHPルートに対して同様のことを行なった時は,2日後にindex登録され,Google検索にもヒットするようになった.
その後,下のように,
Search Console > Google インデックス > ULRの削除 より,削除申請し,indexは削除された.
これのことが何か影響しているのかなぁ....ただ,この削除ってURLごとだし,関係ないと思うのだけれど....
「」の「」にこの記事の続きとして,Search Consoleにサイトマップを登録したときのことをまとめた.
ぼけーっと眺めてて気づいたんですけど,Google Search Consoleでの表示時間って日本時間じゃないんですね....
ログとの照合で16時間足せばいいことがわかったけれど,どこのタイムゾーンを使っているのだろうか....
Search Consoleヘルプ. Google がサポートしているメタタグ. Retrieved August 1, 2017, from https://support.google.com/webmasters/answer/79812?hl=ja |
名前
Email (※公開されることはありません)
コメント