Planet Replay Forum Index Planet Replay
The Destination for ReplayTV Owners and TV Enthusiasts
Back to the home page
 
 FAQFAQ   SearchSearch   7 days of topics7 Days  30 days of topics30 Days  MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

XMLTV Web Scraper Support / Instructions
Goto page 1, 2, 3  Next
 
Post new topic   Reply to topic    Planet Replay Forum Index -> WiRNS
View previous topic :: View next topic  
Author Message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Wed Aug 01, 2007 11:52 pm    Post subject: XMLTV Web Scraper Support / Instructions Reply with quote

This information / process is of interest only to those whose guide data requirements is not be met by Schedules Direct guide data.

The current Updater version of WiRNS includes support for an XML scraper, such as XMLTV.exe or Lava's MSN scraper. In fact, any scraper that supports the XMLTV format.

    1. Install your scraper, set it up, and use it to create an XML output file for each of your lineups. We need the XML file names for the next step.

    2. In the WiRNS installation folder, create a text file named remapProvider.txt (There should be a remapProvider.txt.sample for use as a template).
    This file is used to create "friendly" provider names, and supply some other data if required. The file itself is not 100% necessary, but mostly useful. The format of the file is:
    Scraper provider File Name | Renamed Provider Name | Device (Digital/Analog/etc) | Source Type (Satellite/Cable/Antenna)

    Example:
    ExpressVu-Western Canada4081 XMLOut | Bell ExpressVu - Digital | Digital | Satellite

    New provider File Name: The name of the XML file created by the scraper, WITHOUT THE .XML EXTENSION EXTREMELY IMPORTANT: The scraper creates file names that replaces bad characters, like commas, with spaces. So the file name may well contain 2 consecutive spaces which could easily be mistaken for 1 space. My advice when creating the remap entry is to directly copy the file name from Windows Explorer and paste it into the remap text, rather than typing it.

    Renamed Provider Name: The provider name that will appear in the WiRNS provider dropdowns and on your Replay.

    Device: Digital/Analog.

    Source Type: Satellite/Cable/Antenna

    3. In the same folder where your scraper is installed, create a batch file to execute your scraper. For example, Lava's MSN scraper would require a command like: MSNXMLTV_scraper_V.52.exe /s Settings.xml /d
    Let's assume we named it scraper.bat, in the c:\scraper folder.

    4. Under Configuration / Guide Data
    Enter 0 for Zip Code.
    In the xmlTV command: box, enter the full path to your batch file. eg. C:\scraper\scraper.bat
    Click Update to save your changes.

    5. Initiate a Guide Update - Administration / Update Guide
    This may take a while.

    6. Edit your Replay configurations (Configuration / Replays) and select the new lineup(s) for each Replay. If you're not seeing the new renamed provider names as defined in your remapProvider file, then your remapProvider file is incorrect.

    7. Initiate another Guide Update - Administration / Update Guide
    Sorry, has to be done. You could choose to edit your batch file this one time to not actually execute the scraper, and WiRNS will just reprocess the XML files created earlier. Your choice.

    8. Configure your ReplayTVs to use the new lineup(s).


_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."


Last edited by Glenn1963 on Thu Jun 16, 2011 7:06 am; edited 5 times in total
Back to top
View user's profile Send private message
cliffcor
Planet Master
Planet Master


Joined: 26 Mar 2003
Posts: 664

PostPosted: Thu Aug 02, 2007 4:15 am    Post subject: Reply with quote

Thanks for the step by step Glenn. I'm one of the folks watching intently for the options for the post Zap2It situation. I plan on giving this a go over the weekend.
Cliff
Back to top
View user's profile Send private message
drlava
Replay elitist
Replay elitist


Joined: 22 Sep 2005
Posts: 271

PostPosted: Thu Aug 02, 2007 8:50 am    Post subject: Reply with quote

thanks, glenn. Just a note this is only for WiRNS 2+.
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Thu Aug 02, 2007 2:08 pm    Post subject: Reply with quote

I got MSN_XMLTV scraping data (thanks drlava) so I'll try doing a conversion this weekend too.

ltj
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Fri Aug 03, 2007 6:57 pm    Post subject: Reply with quote

I am following the steps for conversion to MSN_XMLTV but for some reason a frsh install of WiRNS v2.0.0. build 0 won't download a channel guide. It tries but faults, apparently, while getting my channel lineups defined at Zap2it labs.

Here is the error...

[8/3/2007 21:24:19] Checking connection to Zap2it.
[8/3/2007 21:24:21] Backing up database to: C:\Program Files\WiRNS\wirns.db.bak
[8/3/2007 21:24:22] Purging old guide/lineup information
[8/3/2007 21:24:22] Update Time: 2007-08-03 21:24:21
[8/3/2007 21:24:22] Updating guide information in the database.
[8/3/2007 21:24:22] Updating guide for Zap2it account: lonetreejim
[8/3/2007 21:24:22] Downloading guide information for: Fri Aug 03
[8/3/2007 21:24:43] Validating data from Zap2it for: Fri Aug 03
[8/3/2007 21:24:43] Found lineup: Rogers Cable Inc..
[8/3/2007 21:24:43] Found lineup: Rogers Cable Inc. - Digital.

[8/3/2007 21:24:44] Guide Update error: System.NullReferenceException: Object reference not set to an instance of an object.
at WiRNS.WiRNSMain.DoGuideUpdate(Boolean replayStyle) in h:\Main\Development\WiRNS2\WiRNS\WiRNSMain.cs:line 1876

[8/3/2007 21:24:44] An error occured while updating the guide. Restoring database from: C:\Program Files\WiRNS\wirns.db.bak
[8/3/2007 21:24:46] Database restored from: C:\Program Files\WiRNS\wirns.db.bak
[8/3/2007 21:24:46] WiRNS will attempt another guide update in 2 hours.
[8/3/2007 21:24:46] Setting guide update time to : 2007-08-03 23:24:46 for recovery attempt.
[8/3/2007 21:24:46] DNS - Local Host not found

When I load my WiRNS production v1.3.2 Zap2it responds just fine.

Here is the same section of the log for 1.3.2...

[2007-08-03 22:40:44] Guide update manually initiated.
[2007-08-03 22:40:44] Attempting to commence update at 10:40 PM
[2007-08-03 22:40:45] WiRNS v1.3.2 build 0 - (c) 2005 kjac & rbolen70
[2007-08-03 22:40:45] G:\WiRNS\wirns.log has been backed up to G:\WiRNS\wirns.log.20070803224045.txt
[2007-08-03 22:40:45] Backing up database to: G:\WiRNS\wirns.db.bak
[2007-08-03 22:40:47] Purging old guide information
[2007-08-03 22:41:18] Update Time: 2007-08-03 22:40:45
[2007-08-03 22:41:18] Updating guide information in the database.
[2007-08-03 22:41:18] Updating guide for Zap2it account: lonetreejim
[2007-08-03 22:41:18] Processing guide information for: Fri Aug 03
[2007-08-03 22:41:36] Updating lineupmap for: Rogers Cable Inc.
[2007-08-03 22:41:36] Updating lineupmap for: Rogers Cable Inc. - Digital
[2007-08-03 22:41:36] Updating lineupmap for: Cogeco Cable Inc.
[2007-08-03 22:41:36] Updating lineupmap for: Cogeco Cable Inc. - Digital
[2007-08-03 22:41:55] Processing guide information for: Sat Aug 04

Any ideas?
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Fri Aug 03, 2007 7:41 pm    Post subject: Reply with quote

Jim, there's a small possibility it's a remap issue, if you haven't already been there, could you try going to Configuration / Edit Remap File and saving a small change? You can reverse it immediately and save again if you want.

Thanks
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Sat Aug 04, 2007 5:17 am    Post subject: Reply with quote

Glenn,

I uninstalled WiRNS, deleted the WiRNS registry key and then re-installed from scratch using the default options. I updated using the update tool before running WiRNS for the first time last night.

I tried editing the remap file but when I selected it the page I got didn't have any fields to edit. It just listed the headers and the Update button. I even tried renaming the sample but I had no entries in the table.

Oh well, I thought, there must be a way to do it without mapping the old name to the new name and I'm not averse to reselecting my channel plan so I removed the Zap2it entries and just proceeded with the MSN_XMLTV setup. It completed overnight and I used my browser (IE6) to select the proper channel listing for my RTV5040. I updated WiRNS again this moring before proceeding.

Everything looked good at this point. I can see my channel lineup with my browser and I was ready to test my RTV on the test instance of WiRNS.

On my RTV I cleared the channel guide twice and entered the network setup to point my RTV at the new WiRNS test box. It wouldn't verify the network connection. I stopped and restated WiRNS, I tried changing the test box's IP (and used regedit to change the ip WiRNS uses), I turned off the test box's firewall and I'm still unable to verify the network. I pointed the RTV back to the production box ip and it verified in just 5 seconds.

I stopped the test WiRNS, deleted the log and debug files and restared it. Below are the log and debug info.

Can you see where I may have gone wrong?

wirns.log
[8/4/2007 09:00:56] ********************************************
[8/4/2007 09:00:56] Detected that I'm running from command line.
[8/4/2007 09:00:57] Checking Database...
[8/4/2007 09:00:57] Backing up database to: C:\WiRNS\WiRNS.db.startup
[8/4/2007 09:01:01] Backing up dnra.db to: dnra.db.startup
[8/4/2007 09:01:02] Backing up ivsxfer.db to: ivsxfer.db.startup
[8/4/2007 09:01:03] Done Checking Database
[8/4/2007 09:01:04] WiRNS v2.0 build 0 revision 0
[8/4/2007 09:01:04] (c) 2004-2007 rbolen70 and Glenn1963
[8/4/2007 09:01:04] Portions (c) 2003-2004 kjac
[8/4/2007 09:01:04] Thanks: ijprest, waynethedvrguy, Remmer99, gduprey, LeeThompson, j.m., archdog99, FlipFlop, The Dreamer, chriv, hdonzis and all the other Replay Hackers out there.
[8/4/2007 09:01:04] Initializing.
[8/4/2007 09:01:04] Loading configuration
[8/4/2007 09:01:04] Starting WiRNS
[8/4/2007 09:01:04] Configured to update guide at: 8/4/2007 12:45 PM
[8/4/2007 09:01:04] Refreshing Replays.
[8/4/2007 09:01:06] Refreshing RecordingGuide for: Family Room(192.168.2.90)
[8/4/2007 09:01:07] Parsed 63/63 entries.
[8/4/2007 09:01:07] Refreshing PhotoGuide for: Family Room
[8/4/2007 09:01:14] Found 574 files on Family Room
[8/4/2007 09:01:14] Refreshing Replays done.
[8/4/2007 09:01:14] Refreshing ReplayGuide information.
[8/4/2007 09:01:17] Added 152 ReplayGuide shows for Family Room
[8/4/2007 09:01:17] Added 0 ReplayGuide shows for all
[8/4/2007 09:01:17] Refreshing ReplayGuide information done.
[8/4/2007 09:01:17] Checking for shows to hide from Poopli/IVSBase.
[8/4/2007 09:01:18] Checking for shows to hide from Poopli/IVSBase done.
[8/4/2007 09:01:18] Updating ReplayGuide flags.
[8/4/2007 09:01:20] Updating ReplayGuide flags done.
[8/4/2007 09:01:20] Detected configuration settings.
[8/4/2007 09:01:20] Starting Replay Emulation server on: 192.168.2.99:80
[8/4/2007 09:01:20] Loading server plugins.
[8/4/2007 09:01:20] Loading ReplayZone genre data.
[8/4/2007 09:01:20] Plugin: ChannelGuideProvider.dll (2.0.2745.16777) loaded.
[8/4/2007 09:01:20] Plugin: ChannelRemap.dll (2.0.2768.18059) loaded.
[8/4/2007 09:01:20] [PLUGIN] GetNextCall window start is 45 minutes after Guide Update Start Time
[8/4/2007 09:01:20] Plugin: GetNextCall.dll (2.0.2745.16774) loaded.
[8/4/2007 09:01:20] Plugin: GetShellCommands.dll (2.0.2745.16778) loaded.
[8/4/2007 09:01:20] Loading Production Crew Data
[8/4/2007 09:01:20] Configured to refresh ToDo and Replay Guide every 240 minutes.
[8/4/2007 09:01:20] Plugin: GuideServer.dll (2.0.2768.18036) loaded.
[8/4/2007 09:01:20] Plugin: HeadendProvider.dll (2.0.2768.18036) loaded.
[8/4/2007 09:01:20] Plugin: IVS.dll (2.0.2749.33449) loaded.
[8/4/2007 09:01:21] [PLUGIN] IVSProvider loading permanent entries from C:\WiRNS\IVSProvider.conf
[8/4/2007 09:01:21] [PLUGIN] IVSProvider added 0 entries from C:\WiRNS\IVSProvider.conf.
[8/4/2007 09:01:21] [PLUGIN] IVSProvider loading cached entries from C:\WiRNS\IVSProvider.cache
[8/4/2007 09:01:21] [PLUGIN] IVSProvider added 0 entries from C:\WiRNS\IVSProvider.cache.
[8/4/2007 09:01:21] Plugin: IVSProvider.dll (2.0.2745.16778) loaded.
[8/4/2007 09:01:21] Plugin: LoginAuthLogout.dll (2.0.2745.16774) loaded.
[8/4/2007 09:01:21] Plugin: NoPhoneNumbers.dll (2.0.2745.16778) loaded.
[8/4/2007 09:01:21] Plugin: NoServers.dll (2.0.2745.16776) loaded.
[8/4/2007 09:01:21] Plugin: NoSoftwareUpdate.dll (2.0.2745.16778) loaded.
[8/4/2007 09:01:21] Plugin: PutSyslogProvider.dll (2.0.2684.20897) loaded.
[8/4/2007 09:01:21] Plugin: RemoteControl.dll (2.0.2745.16774) loaded.
[8/4/2007 09:01:21] Plugin: RSS.NET.dll (2.0.2745.16775) loaded.
[8/4/2007 09:01:21] Plugin: ServerManagement.dll (2.0.2771.36564) loaded.
[8/4/2007 09:01:21] Plugin: XactProvider.dll (2.0.2745.16776) loaded.
[8/4/2007 09:01:21] Plugin: ZipcodeProvider.dll (2.0.2768.18035) loaded.
[8/4/2007 09:01:21] Replay Network Server Emulation started.
[8/4/2007 09:01:21] Starting configuration server on 127.0.0.1:8923
[8/4/2007 09:01:21] Loading server plugins.
[8/4/2007 09:01:21] Plugin: IVS.dll (2.0.2749.33449) loaded.
[8/4/2007 09:01:21] [PLUGIN] IVSProvider loading permanent entries from C:\WiRNS\IVSProvider.conf
[8/4/2007 09:01:21] [PLUGIN] IVSProvider added 0 entries from C:\WiRNS\IVSProvider.conf.
[8/4/2007 09:01:21] [PLUGIN] IVSProvider loading cached entries from C:\WiRNS\IVSProvider.cache
[8/4/2007 09:01:21] [PLUGIN] IVSProvider added 0 entries from C:\WiRNS\IVSProvider.cache.
[8/4/2007 09:01:22] Plugin: IVSProvider.dll (2.0.2745.16778) loaded.
[8/4/2007 09:01:22] Plugin: ServerManagement.dll (2.0.2771.36564) loaded.
[8/4/2007 09:01:22] Replay Network Server Emulation started.
[8/4/2007 09:01:22] Starting NTP Proxy
[8/4/2007 09:01:22] Starting DNS Proxy
[8/4/2007 09:01:22] **************************************
[8/4/2007 09:01:22] Time Synchronizer (C)2001 Valer BOCAN <vbocan@dataman.ro>
[8/4/2007 09:01:22] Simple Network Time Protocol (see RFC 2030)
[8/4/2007 09:01:22] Modified for WiRNS by rbolen70
[8/4/2007 09:01:22] Connecting to: ntp-production.replaytv.net
[8/4/2007 09:01:22] Local time: 8/4/2007 9:01:22 AM
[8/4/2007 09:01:22] Precision: 8.83423532389192E+74 ms
[8/4/2007 09:01:22] Poll Interval: 16 s
[8/4/2007 09:01:22] Reference ID: 192.168.98.63 (192.168.98.63)
[8/4/2007 09:01:23] Root Dispersion: 48.309326171875 ms
[8/4/2007 09:01:23] Round Trip Delay: 110 ms
[8/4/2007 09:01:23] Local Clock Offset: 73 ms
[8/4/2007 09:01:23] **************************************
[8/4/2007 09:01:23] Remote show sending enabled!
[8/4/2007 09:01:25] Scheduling JIT Recordings
[8/4/2007 09:01:25] Updating ReplayGuide flags.
[8/4/2007 09:01:28] Updating ReplayGuide flags done.
[8/4/2007 09:01:28] Scheduling JIT Recordings complete
[8/4/2007 09:01:28] Ready to intercept/proxy ReplayTV requests


wirns.debug.log
[8/4/2007 09:01:05] HTTPClient: HTTP error connecting to 192.168.2.99
[8/4/2007 09:01:24] DNS - Local Host not found
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Sat Aug 04, 2007 9:32 am    Post subject: Reply with quote

Looks like WiRNS isn't connecting to the DNS Server IP you have defined in Configuration / WiRNS. Any chance there's some sort of issue there?
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Sat Aug 04, 2007 9:55 am    Post subject: Reply with quote

Glenn,

Well, 192.168.2.1 is the IP of my router but, nevertheless, I entered my OpenDNS server address (208.67.222.222) instead and WiRNS restarted without those error messeges.

I have refined my channel selection in the settings.xml (removed channels I don't get to speed up the scrapper) so I'm loading the database before I try to connect my Replay to it.

Thanks for your help. I'll keep you posted.

Jim
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Sat Aug 04, 2007 9:57 am    Post subject: Reply with quote

Jim does bring up a good point, one which I didn't touch on earlier.

It isn't a requirement to follow the TMS remapping instructions above if you don't want to. I implemented the capability only to try to smooth the transition from Zap2it. When successful, you shouldn't have to change anything on your Replay and all your scheduled recordings won't get lost.

But it's not critical to successfully using a scraper as a guide data source. Taking Jim's approach will work (hopefully).

You can and may need to still use the remapProvider.txt file.

You can use it to rename your provider to anything you like, rather than staying with the full XML file name.

You may need to use it to make sure the other provider characteristics - device and source type - are correct.
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Sat Aug 04, 2007 11:29 am    Post subject: Reply with quote

Darn. I stopped and restarted the test WiRNS and I got got errors back .

I uninstalled and started from scratch again and got then again. I changed the DNS to my ISPs default and just got the HTTPClient error.

What are...

HTTPClient: HTTP error connecting to 192.168.2.99

and

DNS - Local Host not found

trying to tell me?
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Sat Aug 04, 2007 4:04 pm    Post subject: Reply with quote

Some success.

I found I had a wrong entry in my wirns.hosts file that I copied to my test setup. It was still pointing me back to the production machine. I fixed it and moved on.

I have my MSN_XMLTV listings but they show up in the satellite listings rather than Cable or Digital. As a result I can't pick the correct settop box remote code.

I setup the remapProvider.txt file to...

Rogers-Kitchener - Digital | Rogers-Guelph, ON - Digital3003 XMLOut | Analog | Cable
Cogeco-Burloak | Cogeco-Burloak, ON3678 XMLOut | Analog | Cable
Cogeco-Burloak - Digital | Cogeco-Burloak, ON - Digital754 XMLOut | Analog | Cable

but that didn't change anything. I also tried...

Rogers-Guelph, ON - Digital3003 XMLOut | Rogers-Kitchener - Digital | Analog | Cable
Cogeco-Burloak, ON3678 XMLOut | Cogeco-Burloak | Analog | Cable
Cogeco-Burloak, ON - Digital754 XMLOut | Cogeco-Burloak - Digital | Analog | Cable

just in case I had them backward but no joy.

I'll try installing the whole thing again.

Jim


EDIT1: BTW, when is remapProvider.txt used? During the RTV connect to WiRNS or when WiRNS builds the database?

EDIT2: I thought the wrong entry in my wirns.hosts was causing my "DNS - Local Host not found" error but apparently not. A brand new clean install produced it using my routers IP. I changed it to my ISPs DNS and restarted and it's OK... for now. I'll see tomorrow if it will hold through a reboot.
Back to top
View user's profile Send private message
drlava
Replay elitist
Replay elitist


Joined: 22 Sep 2005
Posts: 271

PostPosted: Sat Aug 04, 2007 7:26 pm    Post subject: Reply with quote

A lot of your prblems seem to stem from your DNS config, not really this addition.
Also, your second usage of remapProvider is the correct one, except you might want to remove the white spaces around the pipes |.

It is used during the guide update to load the info into the database, not during the RTV connect.
Back to top
View user's profile Send private message
lonetreejim
Replay junkie
Replay junkie


Joined: 14 May 2003
Posts: 161
Location: Kitchener, ON Canada

PostPosted: Sun Aug 05, 2007 8:49 am    Post subject: Success! Reply with quote

I am ready to declare my test to use MSN_XMLTV in place Zap2it Labs for my channel lineup a success. I live in Kitchener, ON (near Toronto) so getting this to work is an absolute must. To make things more interesting, I also serve up the channel listings for my daughters RTV for another service provider in Burlington, ON.

I started off with a spare computer that I had just finished installing Windows XP Pro w/SP2. I used Windows Update to load all the current patches except IE7).

I downloaded and ran MSN_XMLTV, entered my postal code and selected my service provider. I picked my digital cable lineup and added the channels I subscribe to. Interestingly, the service provider MSN_XMLTV was for the next town over (Guelph, ON) but I happen to know they have the same channel lineup so no problem.

When I entered my daughter’s postal code MSN_XMLTV return her service provider’s channel lineup in Niagara Falls, ON about 50 mi from her house. I didn’t know if this was going to be good enough so I looked up postal codes for cities around Burlington and one (Oakville) produced the “BUROAK” lineup for her city. Moral is – if you don’t get your city with you zip, try one close by.

I saved and renamed the settings.xml files for both my and my daughters channel lineups and created a batch file that will call MSN_XMLTV with the settings files as the /s argument. I executed the batch file to see what the names of the .xml files it created would be.

I performed a custom installation of WiRNS 2.0 (v2) turning off the options I don’t use. When the install was finished I updated v2 before running it for the first time.

I started v2 as an application (so I could see what was going on) and it started fine and I accepted the default IP & DNS entries. It discovered my RTV and I used my browser to configure v2 in my “usual” configuration. I entered no Zap2it login information but did point the “xmlTV command:” field to the batch file I created for MSN_XMLTV.

I modified the remapProvider.txt file to give my lineups a less complex name and assign them to be cable lineups. If you don’t do this you can still use it but the lineups will be found in the Satellite providers not the Digital or Analog cable providers (and you’ll need to know the IR code for your set top box). Note: I also use a modified wirns.hosts so my daughters RTV can connect but that doesn’t really impact anything else.

I did a manual update of the guide data. When complete I used IE to associate my RTV with my channel lineup and to turn the “UPNP Server:” off. The latter requires a restart of v2.

When v2 restarted it produced a “DNS - Local Host not found” error in the debug log. This means a bad DNS entry in the “DNS Server IP:” field – but it is my router’s address and should be fine (everything else uses it). On a hunch, I replaced the router’s IP entry with my ISP’s default DNS server IP and restarted v2. It started fine and has been restarted since and it is fine still. I’ve decided to declare this an operating system anomaly for now. I’ll see what happens when I install it on my production box.

I did another manual guide update. When it was done I turned my attention to my RTV.

I used 243-Zones to clear my channel guide and the RTV rebooted. When it was done I cleared the channel guide again. After the 2nd reboot I used the Menu key on the remote to go into the “Setup -> Network and Input Settings” options. From there I chose the “Change Zip Code” option. I entered a different zip code than the one that was there making sure it was in my time zone. I followed the prompts to select the service provider for each input. When I was done I selected “Keep all settings” and the new channel guide loaded.

I checked my channel guide and voila, everything was there! Since I chose this route rather than the one Glenn outlined at the beginning of this thread I will need to go back and recreate all of my timed recording events. The “Theme” recordings seem fine.

Well, that’s it for now. I will convert my RTV back to my Zap2it based WiRNS for now. I am planning deployment of the new version later this month when my daughter and I are both free at the same time.

Thanks to everyone who has been working on this project. I was already pricing a Tivo or two after I heard about Zap2it’s decision but this is way better.

Jim
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Sat Aug 18, 2007 3:15 pm    Post subject: Reply with quote

Had to make a change to allow for non-numeric station ids, so today's WiRNS Updates are a must-have if you're using Lava's scraper.
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
Flavario
Replay elitist
Replay elitist


Joined: 12 Feb 2005
Posts: 372

PostPosted: Mon Aug 20, 2007 8:19 am    Post subject: Reply with quote

Can I use both the scraper and WiRNS method to download listings? or do I have to disable WiRNS downloading of listings by using 0 as my zip code?
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Mon Aug 20, 2007 9:25 am    Post subject: Reply with quote

You can use both, I know Ryan was doing that for PPV
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Mon Aug 27, 2007 12:17 pm    Post subject: Reply with quote

Glenn1963 wrote:
Had to make a change to allow for non-numeric station ids

Had to make another change to WiRNS to handle this. Hopefully it won't mess anybody up...
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Mon Aug 27, 2007 12:31 pm    Post subject: Important note regarding the provider remap file Reply with quote

When creating the provider remap file, remapProvider.txt, it is important to note that the scraper creates file names that replaces bad characters, like commas, with spaces.

So the file name may well contain 2 consecutive spaces, which could easily be mistaken for 1 space.

My advice when creating the remap entry is to directly copy the file name from Windows Explorer and paste it into the remap text, rather than typing it.

For the remap to work correctly, the precise file name must be entered, and a copy/paste is the best way to make that happen.
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
ijprest
Replay newbie
Replay newbie


Joined: 30 Dec 2001
Posts: 20

PostPosted: Sat Sep 01, 2007 9:18 pm    Post subject: Re: XMLTV Web Scraper Support / Instructions Reply with quote

Glenn1963 wrote:
Note: Another text file, tmsids.sav, will be created. This file becomes important as it is a backup of the critical Zap2it data. Don't delete it, and keep a copy of it somewhere else.


I've installed 2.0, and have it successfully (I think) parsing XML data from Lava's MSN scraper. Unfortunately, I was never able to successfully connect to Zap2it after upgrading to 2.0 (even prior to Sept. 1st).

So I guess I'll have to manually create the tmsids.sav file, correct? (It's currently empty, save a couple of lines of comments.)

What's the format of this file? I assume it's just a map from TMSIDs to Lava's channel IDs?

Thanks,
Back to top
View user's profile Send private message
Glenn1963
Planet Master
Planet Master


Joined: 26 Nov 2004
Posts: 1181
Location: Castlegar, BC

PostPosted: Sat Sep 01, 2007 9:42 pm    Post subject: Re: XMLTV Web Scraper Support / Instructions Reply with quote

ijprest wrote:
So I guess I'll have to manually create the tmsids.sav file
Well, it's not that critical. It 's just there to support the zap2it channel ids, but if you never got guide data you wouldn't have had any.

It has no actual effect on the processes, unless you happen to delete your wirns.db, then it would be used to reload the channel ids.

So I wouldn't worry about it.
_________________
G

"I did absolutely nothing, and it was everything that I thought it could be."
Back to top
View user's profile Send private message
ijprest
Replay newbie
Replay newbie


Joined: 30 Dec 2001
Posts: 20

PostPosted: Sat Sep 01, 2007 9:57 pm    Post subject: Reply with quote

OK, no worries. I seem to have everything working, though I'm not 100% sure why.

I only changed two things: First, I renamed one channel in the XML file to use the TMSID. e.g., search and replace "ID242" with "16160". I had expected that this change would allow that one channel to be loaded. However, I also reformatted the XML so that it wasn't all on one (5MB-long) line, which choked all the text-editors I tried for the search/replace.

After those two changes, I updated the guide again and everything started working. I suspect it was the reformatting that made things start working, but I'm not sure why... I wouldn't have expected the .NET XML parser to choke on the file.

Regardless, it's all working now.

Thanks to everyone who made this a reality! I was afraid I was gonna have to dust-off my Linux box and get the old Perl scripts running again!
Back to top
View user's profile Send private message
greenall
Replay fan
Replay fan


Joined: 24 Dec 2002
Posts: 26

PostPosted: Mon Sep 03, 2007 1:13 pm    Post subject: scraper under previous wirns verison Reply with quote

I am trying to get the scraper running under 1.3.2 of wirns. I have setup the scraper, pulled my rogers listing. I am unclear on what I need to do (or even if it is possible) to get 1.3.2 version to recognize the xmltv scraper and input the data.

I can't use the 2.0 version as I'm running windows 2003 x64 on my server, and the 2.0 version won't run on that os.

Any ideas on how I would setup the configuratin in the older wirns version?
Back to top
View user's profile Send private message
drlava
Replay elitist
Replay elitist


Joined: 22 Sep 2005
Posts: 271

PostPosted: Mon Sep 03, 2007 2:34 pm    Post subject: Reply with quote

the newer WiRNS has extra code for the XMLTV import, that code would have to be patched into the 1.3.2 version and recompiled. If you're capable with C# and .net, PM rbolen70 and make a request for code.
Back to top
View user's profile Send private message
greenall
Replay fan
Replay fan


Joined: 24 Dec 2002
Posts: 26

PostPosted: Mon Sep 03, 2007 3:41 pm    Post subject: well, unfortunately, I'm not Reply with quote

Nope, sorry, no experience programming this kind of stuff here. Looks like I'm SOL then unless the problem that keeps 2.0 from running on x64 OS gets fixed.
Back to top
View user's profile Send private message
alman
Replay junkie
Replay junkie


Joined: 20 Dec 2004
Posts: 127
Location: New Brunswick Canada

PostPosted: Sat Sep 08, 2007 5:39 am    Post subject: Adding Missing Channel to a Lineup Reply with quote

After playing into the wee hours of the morning, I'm beginning to see a light at the end of the tunnel with the scraper. Was trying to have my cake and eat it too by trying to use DNNA for Dish and the scraper for cable, and gave up on that thanks to some advice from lonetreejim, and realized that I may even be able to resurrect my Expressvu lineup into the fold using a multiple line xml batch.

Now if I could just figure out how to add a channel or two that are missing from the msn lineup, I may be able to get some sleep tonight.

One of the better half's favorites doesn't show up in the Expressvu-Eastern Canada lineup, even though it's part of the basic Bell subscription.

Channel 199, the Newfoundland CTV affiliate doesn't even show up on the list if I change the postal zip code to the city it originates from.

Is it possible to download this one channel from that city's cable line-up and merge the created XML file into the Bell satellite line-up for my zip code. I see there is a /c option in the command line switches, but that merely combines the channel number & call letters.
________
launch box


Last edited by alman on Thu Feb 03, 2011 1:24 am; edited 1 time in total
Back to top
View user's profile Send private message
drlava
Replay elitist
Replay elitist


Joined: 22 Sep 2005
Posts: 271

PostPosted: Sat Sep 08, 2007 7:49 am    Post subject: Reply with quote

You might be able to do a merge.. have the second line of the remapProviders.txt that references the Bell, second line, end in PPV.
search for the 'ppv trick' to see what's going on.
Back to top
View user's profile Send private message
sanjonny
Planet Master
Planet Master


Joined: 21 Oct 2006
Posts: 877

PostPosted: Sun Sep 09, 2007 4:05 pm    Post subject: Reply with quote

Hey, I think it would be a good idea to start a forum of only scraper issues and move all these threads there since they are tangentially related to wirns and lots of wirns users who don't need to use the scraper can/will get confused.

What do you think admins?
Back to top
View user's profile Send private message Visit poster's website
gavron
Replay fan
Replay fan


Joined: 02 Mar 2002
Posts: 51

PostPosted: Tue Sep 11, 2007 9:52 pm    Post subject: sigh Reply with quote

From the wirns log file:
MetaConnect of Palo Alto(B)CA63068- XMLOut Processing: 100 percent complete.
[9/11/2007 22:45:45] XML Processed 34836 shows.
[9/11/2007 22:45:53] Processing XML File: Comcast Pima County(E)AZ02453X XMLOut
[9/11/2007 22:45:53] Data Source: tv.msn.com
[9/11/2007 22:45:53] Created by: MSN_XMLTV_DownloaderV0.55
[9/11/2007 22:45:53] XML: Processing Lineups
[9/11/2007 22:45:53] XML: Processing Stations
[9/11/2007 22:46:04] Comcast Pima County(E)AZ02453X XMLOut: Processing Schedules for 29887 shows
[

I can't cut and paste from windoze folder view but they are identical. Including dash on the MetaConnect and no dash on Comcast.

from remapProvider.txt
#Provider Name Remap
#Format:
# New Provider File Name | New Provider Name | Device (Digital/Analog/Antenna/etc) | Source Type (Satellite/Cable/Antenna/etc)
#
#ExpressVu-Western Canada4081 XMLOut | Bell ExpressVu - Digital | Digital | Satellite
MetaConnect of Palo Alto(B)CA63068- XMLOut | 4DTV | Digital | Satellite
Comcast Pima County(E)AZ02453X XMLOut | Comcast - Digital | Digital | Cable

What am i doing wrong?

Ehud
Back to top
View user's profile Send private message
gavron
Replay fan
Replay fan


Joined: 02 Mar 2002
Posts: 51

PostPosted: Wed Sep 12, 2007 10:48 am    Post subject: Reply with quote

I have questions... which I've outlined in color...

Where to start.

First, in the beginning, I want to thank everyone who's invested any time and effort into making WiRNS and the MSN_XMLTV scraper what they are today. I understand that the development efforts were rushed by an immediate need to fill a black hole created by Tribune Media Services. I know that "wizards and automagic configuration" things may come later, but the important part was to get things done now. For that I thank you, with no reservations.

Here's what I have working or sorta working:
The ReplayTV is now seeing all the normal lineups, plus the two WiRNS lineups plus one of them with the long-ass XML name. (remapProvider.txt still had .sample after it). One of those lineups (4DTV, aka MetaConnect-XML-whatever) actually has program listings for 4DTV.

Here's what I have left to do:
1. Figure out how remapchan works, so I can remap the MetaConnect lineup to look like Zap2it's 4DTV lineup. Has anyone already done this?

2. Figure out if I want to stick with Tuneitall or get DVRBB. Any suggestions there too? (Which lineup does DVRBB actually want?)

3. Put all of these on one machine, inside a VM on a Linux kernel so that a) it will be reliable, b)I can snapshot it and know it's 100% virus free forever, and c)backup the .VMX files so if the hard drive craps out (like it did last week) I can instantly run the VM on any other machine.


I think that covers it. I've gone in one month from
1. Everything's been working splendidly for years
2. Shoot. Zap2it doesn't work. Upgrade to WiRNS 2.0.
3. Shoot. TuneItAll machine died. Recreate one from scratch.
4. Shoot. Zap2it is dead for good. Use MSN_XMLTV scraper

I know a lot of you were giving me grief that I "only" spent five hours last night and couldn't get it to work. If I were a Windows or C++ developer I would have already spent hundreds more... but I'm not. The only thing I can do is be an end-user... reporting problems I get... solutions I find... and hoping they make sense to developers.

Respectfully, and again let me close by thanking everyone who's gotten this software to the point where it works. (Just because I haven't finished configuring the remapchan doesn't mean i can't see that it WILL work

Ehud
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Planet Replay Forum Index -> WiRNS All times are GMT - 8 Hours
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Planet Replay topic RSS feed 


Powered by phpBB © 2001, 2005 phpBB Group