Metadata-Version: 2.5
Name: clean-dom
Version: 0.2.0
Summary: A lightweight Python library for pruning and cleaning DOM for LLMs
Project-URL: Homepage, https://github.com/AfroditiTzama/clean-dom
Project-URL: Repository, https://github.com/AfroditiTzama/clean-dom
Project-URL: Issues, https://github.com/AfroditiTzama/clean-dom/issues
Author-email: Afroditi Tzama <afro.xama@yahoo.gr>
License: MIT
License-File: LICENSE
Keywords: AI agents,DOM,HTML,LLM,Playwright,Selenium,token optimization,web scraping
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.8
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: lxml>=5.0.0
Provides-Extra: dev
Requires-Dist: build>=1.2.2; extra == 'dev'
Requires-Dist: pyright>=1.1.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
Requires-Dist: pytest>=7.0.0; extra == 'dev'
Requires-Dist: ruff>=0.2.0; extra == 'dev'
Requires-Dist: twine>=5.1.1; extra == 'dev'
Description-Content-Type: text/markdown

# clean-dom

`clean-dom` is a lightweight DOM compression and cleaning layer for LLM and
AI-agent pipelines. It removes configurable HTML noise while retaining useful
HTML structure and semantic attributes for browser agents, web scrapers, and
automation workflows.

Unlike traditional HTML-to-Markdown conversion, which flattens much of the DOM
structure, `clean-dom` prunes the existing HTML representation. This can make
page context more compact while preserving the markup your downstream workflow
needs. The amount of output reduction depends on each page and its content.

## Installation

```bash
pip install clean-dom
```

## Basic Usage

```python
from clean_dom import DOMCleaner

cleaner = DOMCleaner()
cleaned_html = cleaner.clean(raw_html)
```

By default, the cleaner removes common non-content elements such as scripts,
styles, navigation, footers, and embedded frames. It removes HTML comments,
filters attributes to a semantic allowlist, drops `data:` URLs from `src` and
`href`, and removes attribute values longer than 500 characters.

## Removal Rules

```python
cleaner = DOMCleaner(
    additional_remove_tags={"aside", "footer"}
)
```

- `remove_tags` replaces the default removal set. For example,
  `remove_tags={"aside"}` removes only `aside` elements.
- `additional_remove_tags` extends the default removal set, retaining all
  defaults while adding the specified tags.
- `unwrap_tags` removes selected element wrappers while preserving their text
  and child elements.

Use `allowed_attributes` to supply a custom attribute whitelist. Use
`remove_comments=False`, `drop_data_urls=False`, or
`max_attribute_length=None` to disable those respective filters.

## Unwrapping

Unwrapping removes a wrapper without deleting its contents:

```python
cleaner = DOMCleaner(unwrap_tags={"span"})
cleaned_html = cleaner.clean("<p>Hello <span>world</span></p>")
```

The result retains `world` and its place in the paragraph, but not the `span`
element.

## Presets

Choose a built-in configuration with `DOMCleaner.preset(name)`:

- `minimal` keeps only the `id` and `href` attributes, with default tag
  removal.
- `content` removes the default tags plus `aside`, and unwraps `span`.
- `agent` unwraps `span` and preserves the full semantic attribute set,
  including form and accessibility attributes.

For example, the agent preset preserves useful attributes such as `role`,
`aria-label`, `name`, `type`, and `placeholder`:

```python
cleaner = DOMCleaner.preset("agent")
cleaned_html = cleaner.clean(raw_html)
```

## Backwards Compatibility

The `strip_tags` argument and instance attribute remain temporarily supported
for compatibility with `0.1.x`, but are deprecated in favor of
`remove_tags`.

## Safety

`clean-dom` is a DOM pruning/context-optimization library, not an HTML security
sanitizer. It should not replace tools intended for sanitizing untrusted HTML
before browser rendering.

## Development

Install development dependencies with `pip install -e ".[dev]"`. Run tests
with `python -m pytest -v` and lint with `python -m ruff check`.

See [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidance. The project
is distributed under the MIT License; see [LICENSE](LICENSE).
