stormcrawler good first issues (Java)

Delete redirected pages 6 days ago

core good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Remove needsDOM() in ParseFilter 11 days ago

question help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Bug report] Logo not displayed in the documentation 13 days ago

bug help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Keep 'opensearch-java' and remove the older 'opensearch' module? 21 days ago

question help wanted OpenSearch

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

URLPartitionerBolt to partition on arbitrary metadata value 26 days ago

enhancement core good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] Document metadata to be persisted for adaptive scheduling 4 months ago

documentation good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Fetcher: optionally slow down fetching from hosts with repeated exceptions 4 months ago

enhancement core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Send host info on a specific stream 4 months ago

enhancement help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] Replace linear proxy scan in MultiProxyManager with a lookup map 5 months ago

enhancement core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

support retry-after in FetcherBolt 5 months ago

help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

JSoup parser to handle text/plain 5 months ago

enhancement core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Protocol-okhttp: implement IP filter 5 months ago

enhancement core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Fix dead links 5 months ago

documentation help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

New XPATH implementation based on JSON file and organised per hostname or domain 6 months ago

enhancement core good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Capture stats about hosts on status stream post fetching 6 months ago

ideas good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Adaptative URL filter to normalize URLs based on canonical tag 6 months ago

good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

SubDocumentsFilterTest should not use a sitemap file 6 months ago

bug good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Use proxy on a per URL basis 7 months ago

enhancement help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] Documentation: Add PowerShell-specific syntax for Maven archetype generation 7 months ago

documentation good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Move to metrics v2 7 months ago

core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Java 20 deprecates all constructors of the class java.net.URL 8 months ago

core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Configure mockito as an agent 8 months ago

wish good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] The Metadata should handle all the case sensitivities 10 months ago

help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] Work towards Maven 4 Compatibility 12 months ago

help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Make waitAck Cache Configurable in StatusUpdaterBolt about 1 year ago

good first issue OpenSearch

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

MetadataFilter to filter based on multiple key values about 1 year ago

enhancement core help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Decouple Selenium and OkHttp from Core Module about 1 year ago

ideas good first issue Selenium java

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Improve Documentation: Add a Beginner-Friendly Tutorial for Getting Started with SC about 1 year ago

documentation help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

Investigate and fix NOP logger in tests about 1 year ago

help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

[Improvement] Convert Tags in GH actions into SHA over 1 year ago

AI Summary: The task is to update the Apache StormCrawler project's GitHub Actions workflows to use SHA hashes instead of tags for external actions, to comply with Apache's new policy. This involves identifying the external actions used, finding their corresponding SHA hashes, and updating the workflow files accordingly. A tool like eclipse-csi/octopin might assist in this process.

Complexity: 3/5

help wanted good first issue

apache/stormcrawler

982

A scalable, mature and versatile web crawler based on Apache Storm

Java

#apache-storm#crawler#distributed#java#stormcrawler#web-crawler

apache/stormcrawler

Open Issues Need Help