NFPS xmltv EPG Grabber development

Status
Not open for further replies.
#482
What postal code are you using for the canada? I can't get anything using this
Code: 
mc2xml -c ca -g H0H0H0 -C ca.ren -R ca.xml -o ca.xml -D ca.dat -F -s -2 -d 72 -u -a -f

I may have some junk in my ca.chl or ca.ren that may be causing my error...
 
#484
btw ur line is incorrect

mc2xml -c ca -g H0H0H0 -C ca.ren -R ca.xml -o ca.xml -D ca.dat -F -s -2 -d 72 -u -a -f

should be
mc2xml -c ca -g E1V5Y1 -R ca.ren -o ca.xml -C ca.chl -D ca.dat -F -s -2 -d 72 -u -a -f

should work.

and don't forget to strip the .ren file i gave to make the .chl file

.chl
Code: 
WN
I510.45505287.microsoft.com
I845.47494403.microsoft.com
LCN
CICC
CTVED
CTVLH
CKCK
TVA
VHD
CPAC
 
Last edited:
#487
Finally got the canada. Since its local...really didn't care too much about those, so I didn't check the command line for months. :)

Here's what I do with my local...

Code: 
Echo Loading Canadian Lineup
mc2xml -c ca -g e1v5y1 -C ca.chl -R ca.ren -o ca.xml -D ca.dat -F -s -2 -d 72 -u -a -f
sed.exe -e $d ca.xml > ca_strip1.txt
sed.exe -e 1,4d ca_strip1.txt >ca.txt

I do that for each of the pulls, and then combine the *.txt with the header and footer at the end, vs. using the -i process. That way, I'm not dragging the entire xml into the pull for the later big pulls for uk. Normally, I drop the -f, and keep the .xml and let the grabber figure out if it needs to pull new data.
 
#489
This is for Blackbear, so only use this if you know what you are doing. It's my local .bat and the dummy processing that doesn't appear in the github. I also do the webgrab separately from the main pull.
 
#493
Question for Psycon/Dara,

Was looking around webgrab site and came across WG2MP,from the readme...
Code: 
// Small program that replaces the time offset part of the start and stop time in an xmltv file
// by the corrected time string for the local timezone

was thinking about trying to use it instead of xmltv_time_correct but am wonder if you have tried before I changed everything.as it says it only adjusts time I don't know if it will do the date correct also.i'm thinking it would be nice as everything from mc2xml get adjusted to your pc time when imported and wud'nt have to keep adjusting channels2correct to get them right for the webgrab epg part.

any thoughts.

Update

ran some tests with this and it aint going to work it strips the +/- xxxx utc part off which will wreck it for making the nfpsed.xml and nfpsfd.xml files.

thks.
 
Last edited:
#494
not sure.. not a fan of webgrab , its soo damn slow.. I merely incorporated dara's webgrab work into the main grab. i'll bet we can find all channels with mc2xml and be able to remove the need for webgrab all together down the road, which would be ideal IMHO.
 
#495
Go.bat Datemath and sed.exe explained

If you are like me and never used these commands before and wanted to know exactly whats going on well I took the time to do the reading and will try to explain it for those that are running their own grabber.
I had a good idea what it was doing but wanted to know how so here it goes.

first open a cmd prompt and enter

echo %Date:~0,14% and hit enter
you will get a output like this

Tues 08/18/2015

so what happened,well you just asked to display your pc date,thats obvious aint it but what u really asked was show me the pc date starting from position 0 and show the next 14 characters.see the ~0,14% from above.position 0 is at the beginning of the line(to the left of the letter T).
now keeping that in mind lets look at the first commands in that section...

set Day=%Date:~7,2% ----->position 7,next 2 characters = 18 which is todays date and we assign that value to the variable Day via set Day =
set Mth=%Date:~4,2% ----->position 4,next 2 characters = 08 which is the month and we assign that value to the variable Mth via set Mth =
set Yr=%Date:~10,4% ----->position 10,next 4 characters = 2015 which is the year and we assign that value to the variable Yr via set Yr =

so...
Day = 18
Mth = 08
Yr = 2015

next command,i am only going to explain the first one as the rest are just a repeat

call datemath %Yr% %Mth% %Day% + 1 ---->datemath calculates a new date.so we asked to calculate a new date starting from %yr% %Mth% %day% which is 2015 08 18(remember we set these variables above) plus 1 day
set Day1=%_ymd_str% ----> sets the variable Day1 to our newly cacluated date which would be 20150819
the next 2 commads are exactly same but for day2 and day3
so after they are all done we have...

day1 = 20150819
day2 = 20150820
day3 = 20150821

next we make 3 copies of our original dummy file(one for each day we want) and name them dummy1.txt,dummy2.txt,dummy3.txt

finally we want to modify the dates in each of those files for each day of epg
again I will expain the first one as the rest are the same.
sed.exe is nothing more that a editor that we can use to remove,replace,edit text

sed.exe -e "s/20150513213/%Yr%%Mth%%Day%070/g" dummy1.txt > dummy1_1.txt
-e means expression to follow,s means substitute(replace).we want to replace 20150513213(if you look in the dummy.body file you will find this start date) with %Yr%%Mth%%Day%(which we assigned above to be 20150818,todays date.070 sets the time to 0700.g means global which replaces all the starts dates in all our dummy channels in the file named dummy1.txt and outputs the new file as dummy1_1.txt.
sed.exe -e "s/2016051322/%Day1%07/g" dummy1_1.txt > dummy1_conv.txt
this does the exact same thing as above but reads the file Dummy1_1.txt which we created when we replaced the start date but changes the end dates(2016051322,if you look in the dummy.body file you will find this end date) instead of the starting date.the file is outputed as dummy1_conv.txt

so when that's all finished we have

dummy1_conv.txt ---->start date 20150818070000 end date 20150819070000
dummy2_conv.txt ---->start date 20150819070000 end date 20150820070000
dummy3_conv.txt ---->start date 20150820070000 end date 20150821070000

finally we combine all 3 files into one file called dummy_conv.txt
copy /b dummy1_conv.txt + dummy2_conv.txt + dummy3_conv.txt dummy_conv.txt

that's it,now we have 3 days of out dummy channels with correct dates that we add into our epg file.
 
Last edited:
#497
Psycon or anyone good with sed help with this.i am messing with the wg2mp exe I mentioned above.while it reads the + or - utc time and makes the correction to the start/stop time it removes +/- utc value from the xml.
example
Code: 
before
start="20150818070000 +0100" stop="20150818080000 +0100"
after
start="20150818060000" stop="20150818070000"
I want to add +0000 back into the start/stop time as that's what it now corrected for so its like this
Code: 
start="20150818060000 +0000" stop="20150818070000 +0000"
I am trying with this but its giving me a error unterminated s command
Code: 
sed.exe -e "s/(start|stop)="[^"]*/& +0000/g" < "WG2MP_dummy.xml" > "done.xml"
I just don't know sed well enough,never even realized how much it can do,i know it stupid error but can't figure it out.
thks.
 
Status
Not open for further replies.
Top Bottom
Back
Menu