javascript - Chrome 扩展开发。(打字稿)-“TypeError:o 不是函数”仅在打开新选项卡并将 storage.local 设置为“true”时
问题描述
我一直在开发一个用于内部/个人使用的刮刀型 chrome 扩展程序,用于从大学网站上刮取课程数据。
高级算法如下:
- 打开主页,用户可以在其中输入他们想要查找的类数据。(重点是使用此页面中的数据为每个独特的课程页面生成每个可能的 url)
- 生成第一个端点并使用该 url 打开一个新选项卡。(这就是我所说的“二级刮擦”)
- 开始第二度刮擦,完成后,将 设置
chrome.storage.local
为true
。然后关闭选项卡。 - 来自主页的内容脚本读取本地存储并看到状态为真,因此它解决了 Promise。它将本地存储重置为
false
. - 它生成新的 url 并递归地重复该过程,直到创建每个可能的 url。
当我设置存储true
并且从不修改它并且只是 console.log 每个可能的 url 时,扩展运行良好。当我让程序打开一个新选项卡并让它更新时出现错误local.storage
。在使用之前,local.storage
我尝试了使用消息传递(简单且长期存在)和背景的类似实现,但当时我遇到了同样的问题。
关于我可以尝试什么的任何想法?
这是我的代码:
背景/index.ts
chrome.storage.local.set({ secondDegreeState: false });
内容/索引.ts
const un:string = "***";
const pw:string = "***"
const levels:Array<string> = ['L', 'U', 'G'];
let ccyys:string = `20212`;
let search_type_main:string = `FIELD`
let signInSequence:Function = () => {
if(document.getElementById("login-button")){
let signInButton:HTMLInputElement = document.getElementById("login-button")?.children[0] as HTMLInputElement;
let username: HTMLInputElement = document.getElementById("username") as HTMLInputElement;
let password: HTMLInputElement = document.getElementById("password") as HTMLInputElement;
username.value = un;
password.value = pw;
signInButton.value = "Begin Scraping";
setTimeout(() => {
signInButton.click();
console.log('Sign in button pressed');
}, 2000);
}
}
let scrapingSeqence:Function = () => {
if(window.location.href === "https://utdirect.utexas.edu/apps/registrar/course_schedule/20212/"){ // If we are in the main registration page
firstDegreeScrape(0, 1);
}
if(window.location.hostname == "utdirect.utexas.edu"){ // Make sure that we're on a proper hostname
secondDegreeScrape();
}
}
function secondDegreePromise(url:string) : Promise<any> {
/// Open up a new tab with the generated URL
window.open(url, '_blank');
return new Promise (function callback(resolve:Function, reject:Function) {
chrome.storage.local.get(['secondDegreeState'], (response) => {
if(chrome.runtime.lastError) {
console.error(chrome.runtime.lastError.message);
reject("Chrome error");
}else if (response.secondDegreeState === false){ // If the second degree state is still not done
console.log('Still waiting for 2nd degree scrape to finish...'+' Its state is '+response.secondDegreeState);
setTimeout(callback, 5000); // repeat promise after n-seconds until state is true.
}else if(response.secondDegreeState === true){ // If the promise is complete
resolve("2nd degree scrape was complete!");
}else {
reject("Oopsie...");
}
})
});
}
// Two base cases, 1: it reaches the end of the levels array, 2: it reaches the end of the FOS list.
let firstDegreeScrape:Function = (levelNum: number, fosNum: number) => {
// Reset the scrape state (Turns it false)
chrome.storage.local.set({ secondDegreeState: false });
if (levelNum < levels.length){ // If not base case #1
const fosParent:HTMLElement|null = document.getElementById("fos_fl"); // Define the FOS parent element.
if(fosParent){ // If the fosParent is present. (Will most likely return true... just for extra safety)
let fosChildren = fosParent.children;
if(fosNum < fosChildren.length){ // If not base case #2
let fos:HTMLOptionElement = fosChildren[fosNum] as HTMLOptionElement; // The individual field of study.
let fosValue:string = fos.value.split(' ').join('+'); // Format FOS
const url:string = `https://utdirect.utexas.edu/apps/registrar/course_schedule/20212/results/?ccyys=${ccyys}&search_type_main=${search_type_main}&fos_fl=${fosValue}&level=${levels[levelNum]}`;
secondDegreePromise(url)
.then((res)=>{ // If the second degree scrape promise is resolved
console.log(res+"Now moving along to next URL.");
firstDegreeScrape(levelNum, fosNum+1); // Generate the next URL and scrape it
})
.catch(res=>{console.log(res)});
}else {
firstDegreeScrape(levelNum+1, 1);
}
}
}
}
let secondDegreeScrape:Function = () => {
// make sure that there is something to scrape
let table: HTMLTableElement = document.getElementsByClassName('rwd-table')[0] as HTMLTableElement;
if(table){
let t_rows:HTMLCollection = table.children[1].children as HTMLCollection;
let t_rows_arr:Element[] = Array.from(t_rows);
for(let i=0; i < t_rows_arr.length; i++){
// console.log(t_rows_arr[i].childElementCount);
if(t_rows_arr[i].childElementCount == 1){ // If the row is a title
let course_title:any = t_rows_arr[i].childNodes[1].firstChild?.firstChild?.textContent;
let divisionRegex = /^[a-z\s]{0,3}/gi;
let courseNumRegex = /\d*\w/m;
console.log("Division: "+course_title.match(divisionRegex)[0]);
course_title = course_title.replace(divisionRegex, "");
console.log("Course Number: "+course_title.match(courseNumRegex)[0]);
course_title = course_title.replace(courseNumRegex, "");
console.log("Course Name: "+course_title);
}else { // If it's a sub-column
let row = t_rows_arr[i];
let rowChildren = row.childNodes;
let unique = rowChildren[1].childNodes[0].childNodes[0].textContent; //
console.log("Unique: "+unique);
let days = rowChildren[3].textContent;
console.log("Days: "+days);
let hour = rowChildren[5].textContent;
console.log("Hour: "+hour);
// let room;
let instruction_mode = rowChildren[9].textContent;
console.log("Instruction Mode: "+instruction_mode);
let instructor = rowChildren[11].textContent;
console.log("Instructor: "+instructor);
let status = rowChildren[13].textContent;
console.log("Status: "+status);
let flags = rowChildren[15].textContent;
console.log("Flags: "+flags);
let core = rowChildren[17].textContent;
console.log("Core: "+core);
console.log("\n");
}
}
if(document.getElementById("next_nav_link")){ // If there is a next page
setTimeout(()=>{
document.getElementById("next_nav_link")?.click(); // Click the next button
}, 5000)
}else {
setTimeout(()=>{
// Let's complete the 2nd degree scrape (Sets true) & update the local variable
chrome.storage.local.set({ secondDegreeState: true });
//close the tab
window.close();
}, 1000)
}
}
}
let main:Function = () => {
signInSequence();
scrapingSeqence();
}
main();
manifest.json 权限:
- 标签
- 声明性内容
- 贮存
- 活动标签
谢谢您的帮助!
解决方案
推荐阅读
- ruby-on-rails - ruby on rails 教程第 6 章电子邮件验证错误
- javascript - 在组件道具上拥有redux状态副本的正确方法是什么
- html - 如何在html中的按钮下输出这个python变量的结果?
- python - Heroku 使用 --no-cache 选项构建 docker 镜像
- assembly - MIPS - 从堆栈中读取双精度
- reactjs - 您可以使用“常规 Web 应用程序”auth0 应用程序对 React Native 应用程序进行身份验证吗?
- javascript - 如何在 Array javascript 中引用元素的位置。(我没有得到预期的结果)
- sql - 如何在 SQL 中计算日期差异并以天和小时格式显示
- powershell - 如果在 CSV 列中找到术语,则删除行
- amazon-web-services - Amazon Web Services (aws) S3 - 根据主机名限制对存储桶的访问