How to Block Page Element in WebView

[cbanks]

What is the code to block a certain element in a page when using a WebView? I have a web site with ads that I've incorporated into a free app. I want to create a paid app that contains the same site, but blocks the ads from displaying. Thanks.

 

[moster67]

Well, maybe you can use OverRideURL and compare the URL against a pre-defined list of URLs. The URLs you need to check are available by searching Google.

What is the code to block a certain element in a page when using a WebView? I have a web site with ads that I've incorporated into a free app. I want to create a paid app that contains the same site, but blocks the ads from displaying. Thanks.

 

[admac231]

I meant to post this in your other thread but forgot. Sorry.

Use the WebviewExtras library to run javascript that hides the ad container.

If you post the url to the website I can help you out with the javascript.

 

[cbanks]

Hymn 1 | The Morning Breaks

 

[admac231]

Well from the looks of it you can just go ahead and disable javascript.

webview1.JavaScriptEnabled = False

I can't see the use of javascript other than for the ads so that will work fine.

 

[cbanks]

That will work, thanks.

 

[cbanks]

What is the code to do it another way, besides disabling Javascript? I think I may need Javascript to do other things in my app.

 

[cbanks]

Does anyone know how to disable the ads on a page without disabling Javascript? I will need Javascript to perform other functions in my app.

 

[admac231]

I had a crack at it for you. This uses the WebViewExtras library. Hopefully it is easy enough to follow.

Sub Process_Globals

End Sub

Sub Globals
   Dim webView1 As WebView
   webView1.Initialize("webView1")
   Dim webViewExtras1 As WebViewExtras
   webViewExtras1.addJavascriptInterface(webView1,"b4a")
End Sub

Sub Activity_Create(FirstTime As Boolean)
   Activity.AddView(webView1,0,0,100%x,100%y)
   webView1.LoadUrl("http://alumnihighschool.net/scriptures/hymns/1.htm")
   webViewExtras1.addJavascriptInterface(webView1,"B4A")
End Sub

Sub Activity_Resume

End Sub

Sub webView1_OverrideUrl (Url As String) As Boolean
   ProgressDialogShow("Loading...")
End Sub

Sub webView1_PageFinished (Url As String)
   ProgressDialogShow("Loading...")
   webViewExtras1.executeJavascript(webView1,"B4A.CallSub('processHTML',true, document.documentElement.outerHTML)")
End Sub

Sub processHTML(html As String)
   If html.IndexOf("iframe")>-1 Then
      Dim a As Int
      Dim b As Int
      Do While html.IndexOf("iframe") <> -1
         a = html.IndexOf("<iframe")
         b = html.IndexOf2("</iframe>",a)
         html = html.Replace(html.SubString2(a,b+9),"")
      Loop 
      If html.IndexOf("<br><br><br><br>") > -1 Then
         a = html.IndexOf("<br><br><br><br>")
         b = html.IndexOf2("</body>",a)
         html = html.Replace(html.SubString2(a,b),"")
      End If
      webView1.LoadHtml(html)
   End If
   ProgressDialogHide
End Sub

 

[warwound]

I cannot post my reply to this thread - the forum software is blocking it cos it contains some javascript!

Reported the problem here: http://www.b4x.com/forum/bugs-wishlist/14260-vbulletin-blocks-my-post.html#post80756

A text copy of the reply i wanted to post is here: http://code.martinpearman.co.uk/deleteme/b4a_problem_post.txt

Using javascript will be more efficient than admac's solution.

Martin.

 

[cbanks]

warwound: I did what you suggested and it works with "normal" code. But, when I try to remove the "ad" code it still loads the ads, even though that section is removed after the page loads. Any other ideas?

Is there a way to add some code in my app that would just block the ad domain?

Or maybe just insert the ad code at the end of the page after it is done loading?

Or maybe with javascript insert a character somewhere in the domain so that the domain can't be reached?

 

[warwound]

Hi.

I bet this is a timing issue.

The PageFinished event fires after the webpage content has been loaded but BEFORE the javascript that inserts the ads has executed.

If you add a button to your activity and run the code to remove IFRAMEs from the webpage when the button is clicked.
Wait for the page to load and ads appear and click the button - do the ads get removed?

Assuming that works then an idea would be to load the webpage with WebView JavascriptEnabled=False.
Once the PageFinished event fires set JavascriptEnabled=True - you mentioned that you need javascript enabled for your use.

You could also at this point remove all script elements from the page using the method you used to remove iframe elements:

See attached file remove_scripts.txt

But to be honest this is not a solution but a hack!

The real solution is to update the webpages at source on your server - include some way of requesting pages with or without the ads code included.

In another thread you said this is not practical as there are thousands of hand written pages - but it's also not practical to remove the ads from the rendered page.

How is the ads code inserted into all these hand written pages?
Is it possible to remove the ads code from all pages using a decent text editor with a find and replace function?

If you could do that then you need a way to request pages so that ads can be included or not.
If you webserver supports PHP then you could:

1) Make all of your webpages into PHP instead HTM filetypes.

2) Include a single PHP script in each page - this could be done at the same time as you strip the ads code out. Replace the ads code with a little PHP:

<?php
require_once 'path/to/a/php/script.php';
?>

3) The PHP script could check the webpage request URL for a key/value pair and include or not include ads code based on the absence or presence of the key/value pair.

<?php
if(isset($_GET['ads'])){
  if($_GET['ads']!='noads'){
    echo <<<ADSCODE
<script type="text/javascript">
ch_client = "cbanks";
ch_width = 320;
ch_height = 50;
ch_type = "mobile";
ch_sid = "LDS";
</script>
<script src="http://scripts.chitika.net/eminimalls/amm.js" type="text/javascript">
ADSCODE;
  }
}
?>

That's a better solution - you might want to be a little more sophisticated in your choice of name/value used to disable ads but the general technique is the same.

Martin.

 

[cbanks]

warwound: i can't figure out what code to put in my app to determine whether script.php shows the ad code or not. I'm not following the code in #3 well enough.

 

[warwound]

Hi.

Take a look at this page - you'll recognise it from your other thread: Toggle highlight HTML element on click.

Look at the source code of the page and you'll see it contains the ads code.

Load the page again with a key/value pair in the url: Toggle highlight HTML element on click.

Look at the source code and you'll see no ads code.

Here's the two PHP pages saved with .txt file extension so you can see the PHP code:

Toggle highlight HTML element on click

That page contains this:

<?php
require_once 'insert_ads.php';
?>

And here's the text version of that PHP page: http://code.martinpearman.co.uk/deleteme/cbanks/insert_ads.txt

<?php
if(!isset($_GET['ads']) || $_GET['ads']!='no'){
   //   if the webpage has been loaded without an 'ads' key/value pair in the url or the key/value exists but the value is NOT 'no' then insert ad code
   
   //   see here for info on the heredoc syntax i've used: http://www.php.net/manual/en/language.types.string.php#language.types.string.syntax.heredoc
   
    echo <<<ADSCODE
<script type="text/javascript">
ch_client = "cbanks";
ch_width = 320;
ch_height = 50;
ch_type = "mobile";
ch_sid = "LDS";
</script>
<script src="http://scripts.chitika.net/eminimalls/amm.js" type="text/javascript">
ADSCODE;
}
?>

If a key/value is in the url and there is a key named ads with a value of no then no ads code is inserted into the page.
In all other circumstances the ads code WILL be inserted into the page.

If you click a paragarph you might notice a javascript error - the javascript is trying to call a B4A Sub.
Ignore that - i'll be posting a reply on your other thread in a bit..!

Martin.

 

[cbanks]

That makes sense and I can get it to work on my computer now, BUT, my web pages reside on the android device, not on a server, and so when a php page is loaded the php code doesn't run. Any other ideas? Is there a way to not allow traffic from chitika.net?

 

[warwound]

Well if the webpages are on your device then presumably these pages have no ads code in them?

Do you want your app to add ads code if the user has not paid for your app, but if your app has been paid for then you don't want the ads code to be added?

If that's the case then you have two options i think:

1) Load the webpage from your device, if the ads code is required inject it into the page after it has loaded - using the WebView1_PageFinished Sub.

2) You could use a key/value when loading the page from your device - this would work much the same as the PHP method but the code to decide whether to insert the ads code would be in javascript not PHP.

Option 1 means your webpages contain nothing but the webpage content that you wish your users to see.
When a webpage has loaded your activity either injects the javascript into the page or it doesn't.

Option 2 means each page must contain javascript code that looks at the URL used to load the page, looks for the key/value pair and decides whether to write the ads code into the page or not.

Option1 looks best to me - you can add more webpages to your app and not have to bother about whether or not you need to add extra code for the ads - your activity will do all of that with no need to modify the webpage.

Post again with your thoughts and i'll create a little code to show you how either option can be implemented.

Martin.

 

[cbanks]

Yes, I have a free app that I want ads to show. Paid app I want to hide the ads.

I tried injecting Javascript two different ways below and neither one brings up an ad:

Dim ad(9), Jscript As String
ad(0) = "<script Type='text/javascript'>"
ad(1) = "ch_client = 'cbanks';"
ad(2) = "ch_width = 320;"
ad(3) = "ch_height = 50;"
ad(4) = "ch_type = 'mobile';"
ad(5) = "ch_sid = 'LDS';"
ad(6) = "</script>"
ad(7) = "<script src='http://scripts.chitika.net/eminimalls/amm.js' Type='text/javascript'>"
ad(8) = "</script>"
Jscript = ad(0) & ad(1) & ad(2) & ad(3) & ad(4) & ad(5) & ad(6) & ad(7) & ad(8)
WebViewExtras1.executeJavascript(WebView1, Jscript)

WebViewExtras1.executeJavascript(WebView1, "<script Type='text/javascript'>ch_client = 'cbanks';ch_width = 320;ch_height = 50;ch_type = 'mobile';ch_sid = 'LDS';</script><script src='http://scripts.chitika.net/eminimalls/amm.js' Type='text/javascript'></script>")

Is it because I had to change the quotes to ' so that the app would compile?

In addition to helping me figure out what I did wrong to try #1, please show me how I would implement #2. Thanks!

 

[warwound]

Hi.

1) Injecting a <script> tag means more than just trying to use excuteJavascript on the script text...

Here's the javascript you'll want to inject into the page - using executeJavascript:

var ch_client='cbanks',ch_width=320,ch_height=50,ch_type='mobile',ch_sid='LDS',script=document.createElement('script');
script.type='text/javascript';
script.src='http://scripts.chitika.net/eminimalls/amm.js';
document.getElementsByTagName('head')[0].appendChild(script);

See how it creates the variables that the ads script requires then dynamically creates the script tag required for the external javascript file?

That code can be reduced to a single line:

var ch_client='cbanks',ch_width=320,ch_height=50,ch_type='mobile',ch_sid='LDS',script=document.createElement('script');script.type='text/javascript';script.src='http://scripts.chitika.net/eminimalls/amm.js';document.getElementsByTagName('head')[0].appendChild(script);

And the B4A code:

Sub Globals
   Dim ShowAds As Boolean
End Sub

Sub Activity_Create(FirstTime As Boolean)
   ShowAds=True
End Sub

Sub WebView1_PageFinished (Url As String)
   If ShowAds Then
      WebViewExtras1.executeJavascript(WebView1, "var ch_client='cbanks',ch_width=320,ch_height=50,ch_type='mobile',ch_sid='LDS',script=document.createElement('script');script.type='text/javascript';script.src='http://scripts.chitika.net/eminimalls/amm.js';document.body.appendChild(script);")
      Log("Ads code added to webpage")
   End If
End Sub

The page loads then disappears leaving just the ad visible!

2) Use javascript to detect any key/value in the url and insert the ads code:

function getQueryObject(){
   var query=location.search, args={};
   query=query.length>0?query.substring(1):'';
   var items=query.split('&'), item, name, value, i;
   for(i=0; i<items.length; i++){
      item=items[i].split('=');
      name=decodeURIComponent(item[0]);
      value=decodeURIComponent(item[1]);
      args[name]=value;
   }
   return args;
}

//   code to add the ads code based on the key/value ads=no
var query=getQueryObject();
if(!query.ads && query.ads!=='no'){
   //   insert the ads code
   var ch_client='cbanks', ch_width=320, ch_height=50, ch_type='mobile', ch_sid='LDS', script=document.createElement('script');
   script.type='text/javascript';
   script.src='http://scripts.chitika.net/eminimalls/amm.js';
   document.getElementsByTagName('head')[0].appendChild(script);
}

Adding that code to the javascript function which creates the 'highlight on click' listener has the same effect as option 1...
The webpage loads and if ads code is inserted the webpage disappears - the ad generally is visible.

The problem is that the ads script uses document.write to insert itself and ads in the webpage.
If document.write is used while the webpage is being rendered then it does just that - writes some tags or text into the webpage.
BUT if the webpage has already been rendered and then document.write is used the entire rendered page is replaced with whatever document.write writes.

So after the ads code has executed the webpage consists of just this - none of your original webpage remains:

<html>
<head>
<script type="text/javascript" src="http://mm.chitika.net/minimall?&amp;cid=default&amp;screenres=313x483&amp;winsize=480x724&amp;canvas=297x2376&amp;frm=false&amp;history=1&amp;impsrc=amm-1.10.0&amp;url=file%3A///android_asset/my_webpage2.htm&amp;cb=183&amp;loc=8%2C8&amp;snip_title=Toggle%20highlight%20HTML%20element%20on%20click&amp;output=simplejs&amp;callback=ch_ad_render_search"></script>
<script type="text/javascript" src="http://mm.chitika.net/minimall?&amp;w=320&amp;h=50&amp;client=cbanks&amp;sid=LDS&amp;cid=LDS&amp;type=mobile&amp;previous_format=undefinedxundefined&amp;screenres=313x483&amp;winsize=480x724&amp;canvas=313x483&amp;frm=false&amp;history=1&amp;impsrc=amm-1.10.0&amp;url=file%3A///android_asset/my_webpage2.htm&amp;cb=794&amp;loc=8%2C8&amp;snip_title=Toggle%20highlight%20HTML%20element%20on%20click&amp;output=simplejs&amp;callback=ch_ad_render_search"></script>
</head>
<body>
<div id="chitikaAdBeacon-183"></div>
<iframe id="ch_ad183" name="ch_ad183" width="0" height="0" frameborder="0" src="about:blank" marginwidth="0" marginheight="0" vspace="0" hspace="0" allowtransparency="true" scrolling="no"></iframe>
<script type="text/javascript" src="http://scripts.chitika.net/eminimalls/amm.js"></script>
<div id="chitikaAdBeacon-794"></div>
<div></div>
<iframe id="ch_ad794" name="ch_ad794" width="0" height="0" frameborder="0" src="about:blank" marginwidth="0" marginheight="0" vspace="0" hspace="0" allowtransparency="true" scrolling="no" style="display: none; "></iframe>
</body>
</html>

Getting nearer a solution i have updated the B4A code:

'Activity module
Sub Process_Globals
   'These global variables will be declared once when the application starts.
   'These variables can be accessed from all modules.

End Sub

Sub Globals
   'These global variables will be redeclared each time the activity is created.
   'These variables can only be accessed from this module.

   Dim AdParam As String
   Dim ModifiedWebPage, WebPageFilename As String
   Dim UserPaid As Boolean
   Dim WebView1 As WebView
   Dim WebViewExtras1 As WebViewExtras
End Sub

Sub Activity_Create(FirstTime As Boolean)
   ModifiedWebPage=""
   UserPaid=True   '   use this boolean to set whether or not ads should be displayed
   WebPageFilename="my_webpage2.htm"
   
   If UserPaid Then
      AdParam="?ads=no"
   Else
      AdParam=""
   End If
   
   WebView1.Initialize("WebView1")
   WebViewExtras1.addJavascriptInterface(WebView1, "B4A")
   WebViewExtras1.addWebChromeClient(WebView1)
   
   Activity.AddView(WebView1, 0, 0, 100%x, 100%y)
   
End Sub

Sub Activity_Resume
   If File.Exists(File.DirInternalCache, WebPageFilename) Then
      '   GetText assumes the file is encoded using UTF8
      '   the cached webpage will contain no ad code and requires no 'ads' key/value
      WebView1.LoadHtml(File.GetText(File.DirInternalCache, WebPageFilename))
      File.Delete(File.DirInternalCache, WebPageFilename)
      Log("Modified webpage loaded")
   Else
      WebView1.LoadUrl("file:///android_asset/"&WebPageFilename&AdParam)
      Log("Original webpage loaded")
   End If
End Sub

Sub Activity_Pause (UserClosed As Boolean)
   If ModifiedWebPage="" Then
      Log("The webpage contains no highlighted elements so there is no need to save it")
   Else
      Log("The webpage has been modified - cache it to the filesystem")
      File.WriteString(File.DirInternalCache, WebPageFilename, ModifiedWebPage)
   End If
End Sub

Sub SaveHtml(Html As String)
   '   this Sub is called only by the WebView1 click event listener
   '   Html will be either the entire modified webpage html or (if the webpage is NOT modified) an empty String
   '   Log(Html)
   ModifiedWebPage=Html
End Sub

And the webpage code:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Toggle highlight HTML element on click</title>
<style type="text/css">
.highlight{
   background-color: black;
   color: white;
}
</style>
<script type="text/javascript">
function createListener(){
   //   code to add the highlight on click event listener
   document.addEventListener('click', function(event){
      var element=event.target;
      var tagName=element.tagName.toLowerCase();
      //   detect HTML tags that should NOT be highlighted
      if(tagName!=='html' && tagName!=='body' && tagName!=='a'){
         if(element.className===''){
            element.className='highlight';
         } else {
            element.className='';
         }
      }
      var modifiedElements=document.getElementsByClassName('highlight'), text;
      if(modifiedElements.length>0){
         //   some elements have been highlighted so send the entire modofied page to a B4A Sub SaveHtml
         text=document.documentElement.outerHTML;
      } else {
         //   the page currently contains no highlighted elements send an empty string to the B4A Sub SaveHtml to indicate there is no need to save the page
         text='';
      }
      B4A.CallSub('SaveHtml', true, text);
   }, false);
}
</script>
</head>
<body onload="createListener()">
<p> Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed venenatis, leo et ornare tempor, est tellus suscipit mauris, et tincidunt lectus justo eu mauris. Vivamus sit amet est id lorem sodales semper. Integer odio nisi, aliquet in tincidunt vitae, auctor vel nisi. Phasellus lacinia velit eu ligula elementum commodo viverra sapien sodales. Duis tincidunt est arcu, ut facilisis arcu. Nullam pulvinar interdum aliquet. Suspendisse eleifend convallis lacinia. </p>
<p> Aenean sed mi justo, commodo imperdiet urna. Praesent adipiscing, nisl sit amet viverra tempor, arcu nisl imperdiet libero, ut vestibulum lorem nisl at erat. Aliquam tincidunt suscipit diam, sit amet blandit ligula ultricies eget. Suspendisse suscipit enim eget ante ultricies vel laoreet est commodo. Duis placerat sem id libero aliquam auctor. Vestibulum sed quam ipsum, ac volutpat ante. Cras fringilla elit in nunc lobortis bibendum varius dolor ultricies. Nullam est mi, bibendum vitae tempus vitae, aliquet non neque. Sed urna sapien, lobortis non molestie ac, gravida nec ligula. </p>
<script type="text/javascript">
function getQueryObject(){
   var query=location.search, args={};
   query=query.length>0?query.substring(1):'';
   var items=query.split('&'), item, name, value, i;
   for(i=0; i<items.length; i++){
      item=items[i].split('=');
      name=decodeURIComponent(item[0]);
      value=decodeURIComponent(item[1]);
      args[name]=value;
   }
   return args;
}
//   code to add the ads code based on the key/value ads=no
var query=getQueryObject();
if(!query.ads || query.ads!=='no'){
   //   insert the ads code
   query=null;   //   no longer need to keep a reference to this object
   var ch_client='cbanks', ch_width=320, ch_height=50, ch_type='mobile', ch_sid='LDS';
   document.write('<script type="text/javascript" src="http://scripts.chitika.net/eminimalls/amm.js"></sc'+'ript>');
   console.log('ads code inserted');
} else {
   console.log('ads code NOT inserted');
}
</script>
</body>
</html>

Now everything works as intended BUT on an orientation change, if a cached webpage is reloaded there is no way to send to add the AdParam parameter so ads will always be displayed on orientation change.
There is no way to add the AdParam String to this line in B4A:

WebView1.LoadHtml(File.GetText(File.DirInternalCache, WebPageFilename))

In B4A we can use file:///android_asset/filename in the WebView LoadUrl method but we cannot (i think but could be wrong) create a path using something like:

WebView1.LoadUrl(File.DirInternalCache&WebPageFilename&AdParam)

Another problem i thought of - what if a user of your free app upgrades and pays - any saved modified webpages WILL contained the ads code!

So i think we have the basics of working code to insert or not insert ads code and also highlight page elements and save modified pages - what you have to do now is think how best to (re)structure your application so these bits of code can be added without the problems i mentioned above.

Are you adding (a lot of) webpages to your application - lots of webpages in the android asset folder?
Saving a copy of each modified webpage is gonna start using a lot of memory.
You could save the modified webpages to external storage instead of the app's Internal or InternalCache folder.
This would then allow you to load modified webpages using the LoadUrl method and add the AdParam String - i think you can use something like file:///sd-ext/foldername/webpage.htm?ads=no but you'd have to check that that is valid.

But surely at some point you'll want to add so many webpages to your application assets that the APK size will get ridiculous?

Have you thought about using a database to store ALL pages?

A simple table with two columns: 'filename' and 'content' could be used.
When you create you app you could manually add some webpages to the database.
Add as many as you require but not that many that your APK is bloated.

When a webpage is modified just query the database and UPDATE the content for that webpage.

If a user requires a webpage that is not currently in the database you could use HttPClient (or HttpUtils if you want a ready made code module) to download the webpage from the internet.
Once downloaded you can INSERT it into the database and display it in the WebView.

Just looking at your other post where you say this app is basically a bookreader.
Could you create a database for each book - a single database containing all webpages for that book.

You could create you app with no database included and then offer the user a choice of books to download from the internet.
You'd need some webhosting but if you have that then your app is much more flexible.
Maybe the app could download more than one book - each book being either a database to itself or a database table to itself.

If you forget about the ads code and concentrate more on the webpages and storage of the webpages then once you have that working look at adding the ads code back to your new code.

Just some ideas - hope i haven't confused things too much.

Martin.