{ "version": "1.0.0", "exported_at": "2026-06-01T10:30:00.000Z", "project": { "name": "Google Shopping Price Monitor by URL", "description": "Best-effort Google Shopping price monitor equivalent to the Octoparse template. Scrapes Google Shopping product/search URLs supplied in navigate.urls[], extracting keyword, product name, Google product URL, rating, review count, price, original price, discount information, store, product source URL, delivery/returns, and product image. Navigation uses a multi-URL input loop with loop-continue so multiple Google Shopping URLs can be processed in one run. If Google blocks the page or no product cards are detected, the template exports one fallback row for the URL with available keyword/title context and blank price fields instead of producing no output. Google may block scraping with browser-update, CAPTCHA, consent, or anti-bot pages.", "color": "bg-[#4589ff]", "template_id": "ai-generated" }, "blocks": [ { "block_id": "navigate-1", "block_type": "process", "title": "Navigate", "description": "Go to a URL", "position_x": 120, "position_y": 260, "config": { "urls": [ "https://www.google.com/shopping/product/8536363455278739214?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:10202490487543604026_0,pid:12371474594059836764,rsk:PC_5729469329925759235&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wII1xM", "https://www.google.com/shopping/product/11765966297896664613?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:17587585761774523052_0,pid:13085054312283802831,rsk:PC_766687629029581349&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wII7hM", "https://www.google.com/shopping/product/6720013340173708684?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:7487761123012911373_0,pid:7214002723375226750,rsk:PC_766687629029581349&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wII_xM", "https://www.google.com/shopping/product/6721531415187866981?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:10953905873025505984_0,pid:6120426756181361799,rsk:PROD_VAR_308239486139304444&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wIIkxQ", "https://www.google.com/shopping/product/12007748746335941851?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:12339129827695444439_0,pid:8638895092709132339,rsk:PC_5854979780296652879&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wIIpRQ", "https://www.google.com/shopping/product/9392410312489069990?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:406245465017959081_0,pid:2406075589709346635,rsk:PC_14965957521147122846&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wIIuhQ", "https://www.google.com/shopping/product/2025859573748970423?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:2124843532480845128_0,pid:92903104801834605,rsk:PC_13097011623620988967&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wIIyBQ", "https://www.google.com/shopping/product/12550956628690432373?hl=en&psb=1&gl=us&q=iPad&oq=iPad&gs_lcp=Cgtwcm9kdWN0cy1jYxAMMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABFAKWApg_AtoAnAAeACAAUKIAUKSAQExmAEAoAEBsAEA&sclient=products-cc&prds=eto:1992057622721314989_0,pid:8838620447949762591,rsk:PC_12254798327478108634&sa=X&ved=0ahUKEwiXy6jZ4KSCAxUTATQIHeV0Bs0Q8wII1xQ" ], "color": "bg-[#4589ff]", "tags": [ "google-shopping", "multi-url", "best-effort" ] } }, { "block_id": "wait-for-page-load-1", "block_type": "process", "title": "Wait for Page Load", "description": "Wait for page to finish loading", "position_x": 480, "position_y": 260, "config": { "timeout": 30 } }, { "block_id": "sleep-1", "block_type": "process", "title": "Sleep", "description": "Wait for specified time", "position_x": 840, "position_y": 260, "config": { "duration": 3 } }, { "block_id": "inject-javascript-1", "block_type": "process", "title": "Inject JavaScript", "description": "Execute custom JavaScript", "position_x": 1200, "position_y": 260, "config": { "jsCode": "(function(){const priceRe=/\\$\\s?[0-9][0-9,]*(?:\\.[0-9]{2})?/;document.querySelectorAll('[data-uscraper-row=google-shopping-product]').forEach(e=>e.removeAttribute('data-uscraper-row'));document.body.removeAttribute('data-uscraper-fallback-row');const candidates=Array.from(document.querySelectorAll('div,li,article,g-card,[role=listitem]'));const usable=candidates.map(el=>{const text=(el.innerText||'').replace(/\\s+/g,' ').trim();if(!text||text.length<20||text.length>1800)return null;const links=Array.from(el.querySelectorAll('a[href]'));const hasPrice=priceRe.test(text);const hasImg=!!Array.from(el.querySelectorAll('img[src]')).find(i=>i.src&&!i.src.startsWith('data:'));const hasGoogleProduct=links.some(a=>/\\/shopping\\/product\\//.test(a.href)||/prds=/.test(a.href));const hasOutbound=links.some(a=>a.hostname&&!/google\\.|gstatic\\.|encrypted-tbn/i.test(a.hostname));const hasRating=/[0-5](?:\\.\\d)?\\s*(?:out of 5|stars?)|[0-9,]+\\s+reviews?/i.test(text);const score=(hasPrice?3:0)+(hasImg?1:0)+(hasGoogleProduct?2:0)+(hasOutbound?2:0)+(hasRating?1:0);return score>=3?{el,text,score}:null;}).filter(Boolean).sort((a,b)=>a.text.length-b.text.length);const selected=[];for(const item of usable){if(selected.some(s=>s.contains(item.el)||item.el.contains(s)))continue;selected.push(item.el);}if(selected.length){selected.slice(0,100).forEach(el=>el.setAttribute('data-uscraper-row','google-shopping-product'));return selected.length;}document.body.setAttribute('data-uscraper-row','google-shopping-product');document.body.setAttribute('data-uscraper-fallback-row','true');return 1;})()", "waitForCompletion": true, "timeout": 10 } }, { "block_id": "wait-for-element-1", "block_type": "process", "title": "Wait for Element", "description": "Wait until element appears", "position_x": 1560, "position_y": 260, "config": { "selector": "[data-uscraper-row='google-shopping-product']", "timeout": 10, "visible": false } }, { "block_id": "structured-export-1", "block_type": "process", "title": "Structured Export", "description": "Export data with custom columns", "position_x": 1920, "position_y": 260, "config": { "rowSelector": "[data-uscraper-row='google-shopping-product']", "fileName": "google-shopping-price-monitor-by-url.csv", "saveLocation": "C:\\Users\\theskd\\Documents\\UScraper\\templates", "includeHeaders": true, "fileMode": "append", "columns": [ { "name": "Keyword", "selector": "new URLSearchParams(location.search).get('q') || new URLSearchParams(location.search).get('oq') || ''", "attribute": "text", "isJs": true }, { "name": "ProductName", "selector": "(() => { const fallback = ROW.getAttribute('data-uscraper-fallback-row') === 'true'; if (fallback) return document.querySelector('meta[property=\"og:title\"],meta[name=\"twitter:title\"]')?.content || document.title || ''; const pick = ROW.querySelector('h3,[role=heading],.tAxDx,.sh-np__product-title'); if (pick && pick.textContent.trim()) return pick.textContent.replace(/\\s+/g,' ').trim(); const texts = Array.from(ROW.querySelectorAll('a,span,div')).map(e => (e.textContent||'').replace(/\\s+/g,' ').trim()).filter(t => t.length>8 && !/^\\$/.test(t) && !/reviews?|stars?|delivery|returns|sponsored|shop now/i.test(t)); return texts[0] || document.querySelector('meta[property=\"og:title\"],meta[name=\"twitter:title\"]')?.content || document.title || ''; })()", "attribute": "text", "isJs": true }, { "name": "ProductURL", "selector": "(() => { const a = Array.from(ROW.querySelectorAll('a[href]')).find(a => /\\/shopping\\/product\\//.test(a.href)); return a ? a.href : location.href; })()", "attribute": "text", "isJs": true }, { "name": "Rating", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const aria = Array.from(ROW.querySelectorAll('[aria-label]')).map(e => e.getAttribute('aria-label')).find(t => /[0-5](?:\\.\\d)? out of 5|stars/i.test(t || '')); const t = (aria || ROW.innerText || '').replace(/\\s+/g,' '); const m = t.match(/([0-5](?:\\.\\d)?)\\s*(?:out of 5|stars?)/i) || t.match(/\\b([0-5]\\.\\d)\\b/); return m ? m[1] : ''; })()", "attribute": "text", "isJs": true }, { "name": "ReviewCount", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const t = (ROW.innerText || '').replace(/\\s+/g,' '); const m = t.match(/([0-9][0-9,]*)\\s*(?:product\\s*)?reviews?/i) || t.match(/\\(([^)]*[0-9][0-9,]*)\\)/); return m ? m[1].replace(/[^0-9,]/g,'') : ''; })()", "attribute": "text", "isJs": true }, { "name": "Tags", "selector": "(() => { const fallback = ROW.getAttribute('data-uscraper-fallback-row') === 'true'; if (fallback) { const text = (document.body.innerText || '').replace(/\\s+/g,' ').trim(); if (/update your browser/i.test(text)) return 'Blocked: update your browser page'; if (/captcha|unusual traffic|verify/i.test(text)) return 'Blocked: CAPTCHA or anti-bot page'; return 'Fallback row: no product card detected'; } const vals = Array.from(ROW.querySelectorAll('[aria-label]')).map(e => e.getAttribute('aria-label')).filter(t => t && /(stars?|reviews?|out of 5)/i.test(t)); return Array.from(new Set(vals)).join(' | '); })()", "attribute": "text", "isJs": true }, { "name": "Price", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const t = (ROW.innerText || '').replace(/\\s+/g,' '); const m = t.match(/\\$\\s?[0-9][0-9,]*(?:\\.[0-9]{2})?/); return m ? m[0] : ''; })()", "attribute": "text", "isJs": true }, { "name": "OriginalPrice", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const t = (ROW.innerText || '').replace(/\\s+/g,' '); const prices = Array.from(t.matchAll(/\\$\\s?[0-9][0-9,]*(?:\\.[0-9]{2})?/g)).map(m => m[0]); return prices.length > 1 ? prices[1] : ''; })()", "attribute": "text", "isJs": true }, { "name": "DiscountInfomation", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const lines = (ROW.innerText || '').split(/\\n+/).map(s => s.trim()).filter(Boolean); return lines.find(s => /(discount|apply|save|off|coupon|promo)/i.test(s)) || ''; })()", "attribute": "text", "isJs": true }, { "name": "Store", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const a = Array.from(ROW.querySelectorAll('a[href]')).find(a => a.href && !/google\\.|gstatic\\.|encrypted-tbn/i.test(a.hostname) && (a.textContent || '').trim()); if (a) return a.textContent.replace(/\\s+/g,' ').trim(); const el = ROW.querySelector('.aULzUe,.IuHnof,.E5ocAb'); return el ? el.textContent.replace(/\\s+/g,' ').trim() : ''; })()", "attribute": "text", "isJs": true }, { "name": "ProductSourceURL", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const direct = Array.from(ROW.querySelectorAll('a[href]')).find(a => a.href && !/google\\.|gstatic\\.|encrypted-tbn/i.test(a.hostname)); if (direct) return direct.href; const ga = Array.from(ROW.querySelectorAll('a[href]')).find(a => /\\/url\\?/.test(a.href) || /[?&](url|q)=https?/i.test(a.href)); if (!ga) return ''; try { const u = new URL(ga.href); return u.searchParams.get('url') || u.searchParams.get('q') || ga.href; } catch(e) { return ga.href; } })()", "attribute": "text", "isJs": true }, { "name": "DeliveryAndRerurns", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const lines = (ROW.innerText || '').split(/\\n+/).map(s => s.trim()).filter(Boolean); return lines.find(s => /(delivery|shipping|returns?)/i.test(s)) || ''; })()", "attribute": "text", "isJs": true }, { "name": "ProductImage", "selector": "(() => { if (ROW.getAttribute('data-uscraper-fallback-row') === 'true') return ''; const img = Array.from(ROW.querySelectorAll('img[src]')).find(i => i.src && !i.src.startsWith('data:')); return img ? img.src : ''; })()", "attribute": "text", "isJs": true } ] } }, { "block_id": "loop-continue-1", "block_type": "process", "title": "Loop Continue", "description": "Continue multi-input loop", "position_x": 2280, "position_y": 260, "config": {} } ], "connections": [ { "from_block_id": "navigate-1", "from_connector_id": "right", "to_block_id": "wait-for-page-load-1", "to_connector_id": "left" }, { "from_block_id": "wait-for-page-load-1", "from_connector_id": "right", "to_block_id": "sleep-1", "to_connector_id": "left" }, { "from_block_id": "sleep-1", "from_connector_id": "right", "to_block_id": "inject-javascript-1", "to_connector_id": "left" }, { "from_block_id": "inject-javascript-1", "from_connector_id": "right", "to_block_id": "wait-for-element-1", "to_connector_id": "left" }, { "from_block_id": "wait-for-element-1", "from_connector_id": "right", "to_block_id": "structured-export-1", "to_connector_id": "left" }, { "from_block_id": "structured-export-1", "from_connector_id": "right", "to_block_id": "loop-continue-1", "to_connector_id": "left" } ], "canvas_elements": [ { "id": "group-load", "element_type": "group", "title": "Page Load", "color": "#08bdba", "position_x": 48, "position_y": 156, "width": 1760, "height": 296, "z_index": 20, "data": { "memberBlockIds": [ "navigate-1", "wait-for-page-load-1", "sleep-1", "wait-for-element-1" ] } }, { "id": "group-interaction", "element_type": "group", "title": "Interaction", "color": "#a56eff", "position_x": 1128, "position_y": 156, "width": 380, "height": 296, "z_index": 20, "data": { "memberBlockIds": [ "inject-javascript-1" ] } }, { "id": "group-extract", "element_type": "group", "title": "Data Extraction", "color": "#42be65", "position_x": 1848, "position_y": 156, "width": 380, "height": 296, "z_index": 20, "data": { "memberBlockIds": [ "structured-export-1" ] } }, { "id": "group-pagination", "element_type": "group", "title": "Pagination Loop", "color": "#ff832b", "position_x": 2208, "position_y": 156, "width": 380, "height": 296, "z_index": 20, "data": { "memberBlockIds": [ "loop-continue-1" ] } }, { "id": "note-overview", "element_type": "note", "title": "Overview", "content": "Best-effort Google Shopping price monitor equivalent to the Octoparse template. Scrapes Google Shopping product/search URLs supplied in navigate.urls[], extracting keyword, product name, Google product URL, rating, review count, price, original price, discount information, store, product source URL, delivery/returns, and product image. Navigation uses a multi-URL input loop with loop-continue so multiple Google Shopping URLs can be processed in one run. If Google blocks the page or no product cards are detected, the template exports one fallback row for the URL with available keyword/title context and blank price fields instead of producing no output. Google may block scraping with browser-update, CAPTCHA, consent, or anti-bot pages.", "color": "#f1c21b", "position_x": 80, "position_y": 20, "width": 480, "height": 160, "z_index": 22, "data": {} }, { "id": "note-block-navigate-1", "element_type": "note", "title": "Note: Navigate", "content": "Multi-URL loop over 8 pages. Pair with loop-continue at the end of each iteration.", "color": "#ee5396", "position_x": 320, "position_y": 240, "width": 328, "height": 107, "z_index": 22, "data": { "block_id": "navigate-1" } }, { "id": "note-block-inject-javascript-1", "element_type": "note", "title": "Note: Inject JavaScript", "content": "Runs custom JavaScript in the page: `(function(){const priceRe=/\\$\\s?[0-9][0-9,]*(?:\\.[0-9]{2})?/;document.querySelectorAll('[data-uscrap...` Verify in browser if results are empty.", "color": "#ee5396", "position_x": 1400, "position_y": 240, "width": 340, "height": 140, "z_index": 22, "data": { "block_id": "inject-javascript-1" } }, { "id": "note-block-structured-export-1", "element_type": "note", "title": "Note: Structured Export", "content": "Structured export with JS columns (Keyword, ProductName, ProductURL, Rating, ReviewCount). These selectors are fragile — update if the site layout changes.", "color": "#ee5396", "position_x": 2120, "position_y": 240, "width": 340, "height": 131, "z_index": 22, "data": { "block_id": "structured-export-1" } }, { "id": "note-block-loop-continue-1", "element_type": "note", "title": "Note: Loop Continue", "content": "Loop Continue advances a multi-URL or multi-text loop. Place at the end of the loop body with a clear back-edge to the loop start.", "color": "#ee5396", "position_x": 2480, "position_y": 240, "width": 340, "height": 123, "z_index": 22, "data": { "block_id": "loop-continue-1" } } ] }