mattdarnell Member Licensed User Longtime User Dec 6, 2013 #1 Aloha, I am trying to scrape this website - https://btc-e.com/exchange/xpm_btc Unfortunately when I open it using the http2 tool I don't get the main page, I get a google apps page. Is there a way to have the http2 tool redirect to the main page or do what ever a browser does so I can scrape some of the data for my app? Mahalo in advance. Aloha, Matt
Aloha, I am trying to scrape this website - https://btc-e.com/exchange/xpm_btc Unfortunately when I open it using the http2 tool I don't get the main page, I get a google apps page. Is there a way to have the http2 tool redirect to the main page or do what ever a browser does so I can scrape some of the data for my app? Mahalo in advance. Aloha, Matt
NJDude Expert Licensed User Longtime User Dec 6, 2013 #2 Why scrape? they have an API, look at the bottom of that page. Upvote 0
mattdarnell Member Licensed User Longtime User Dec 6, 2013 #3 NJDude, didn't even see that, I will check it out! For the record, this is what you see when you bring up the page with t he http2 tool: <html><head><title>loading</title></head><body><p>Please wait...</p><script>document.cookie="a=c72b8a4fa37d475e6900a5165a1f6932;path=/;";location.href=document.location.pathname;</script><script>var _gaq = _gaq || []; _gaq.push(["_setAccount", "UA-25049930-1"]); _gaq.push(["_trackPageview"]); (function(){var ga = document.createElement("script"); ga.type = "text/javascript"; ga.async = true;ga.src = ("https:" == document.location.protocol ? "https://ssl" : "http://www") + ".google-analytics.com/ga.js";var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(ga, s);})();</script></body></html> Upvote 0
NJDude, didn't even see that, I will check it out! For the record, this is what you see when you bring up the page with t he http2 tool: <html><head><title>loading</title></head><body><p>Please wait...</p><script>document.cookie="a=c72b8a4fa37d475e6900a5165a1f6932;path=/;";location.href=document.location.pathname;</script><script>var _gaq = _gaq || []; _gaq.push(["_setAccount", "UA-25049930-1"]); _gaq.push(["_trackPageview"]); (function(){var ga = document.createElement("script"); ga.type = "text/javascript"; ga.async = true;ga.src = ("https:" == document.location.protocol ? "https://ssl" : "http://www") + ".google-analytics.com/ga.js";var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(ga, s);})();</script></body></html>
Erel B4X founder Staff member Licensed User Longtime User Dec 7, 2013 #4 If they offer an API then you should use it. As a tip you can use FireBug to see the network calls and then imitate them in your code. Upvote 0
If they offer an API then you should use it. As a tip you can use FireBug to see the network calls and then imitate them in your code.
