For the analysis of a log file I have a CSV-file from this log error codes (sorted ascending on date and time). I need to find a certain event code (for example ‘4797’; yellow in the screenshot). From there I need to look back in time for a specific preceding error code (for example ‘268’; orange in the screenshot) and extract all of these and their time stamps.
The output should be each event code with time stamp and all error codes with time stamps preceding that specific one.
for each event code 4797:
get all error codes 234
I can find and extract the event codes and time stamps by reading the csv-file and appending each row to a list of lists and looking for the error code at the correct element.
How can I look back from that point to find the connected preceding error code ( and repeat it without extracting the same error code twice)?
Should I look for the event code, cut the list prior to this and look for the error code in the smaller list (and repeat this until there are no more event codes)? This sounds cumbersome, but I cannot think of a way to do this…
If the csv is already sorted by the date: open it with Pandas, get the ID for the row with error-code 4797, slice it from 0 to that row and get all the rows in the slice with error-code 234.
From Jagaya, Pandas dataframe is a good tool for data selecting. Maybe can help you to solve the problem.
Lots of thanks!
I have this working for multiple error codes 4797 by first making a df, then make a list from the df with all ID’s for all rows with 4797, then iterating through this and by slicing creating a new df for each part from the previous 4797 until the next and select all codes 234 (and some others while I was at it ). I will refine and test some more and then probably find more challenges to solve with this
Yes, you are right! I thought about it and Jagaya’s clear description of what to do it helped me.
Did you finish your scraper and built a nice database of properties in the mean time?
Not yet, I try another web site which is updated more frequency. But I face another problem on Chinese Characters handling on html source page, I can see it from web browser, but missing those Chinese characters from extraction. Maybe it is a encode/decode problem. I have posted this question on forum too
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.