Skip to content

chunklet.visualizer.visualizer

Classes:

  • Visualizer –

    A FastAPI-based web interface for visualizing document and code chunks.

Visualizer 

Visualizer(
    host: str = "127.0.0.1",
    port: int = 8000,
    token_counter: Callable[[str], int] | None = None,
)

A FastAPI-based web interface for visualizing document and code chunks.

This server allows users to upload text or code files, processes them with Chunklet's DocumentChunker or CodeChunker, and returns the chunked data along with statistics. A minimal frontend interface is served at the root endpoint.

Attributes:

  • host (str) –

    Host IP to bind the FastAPI server.

  • port (int) –

    Port number to run the server on.

  • document_chunker (DocumentChunker) –

    Chunklet document chunker instance.

  • code_chunker (CodeChunker) –

    Chunklet code chunker instance.

  • app (FastAPI) –

    FastAPI application instance.

Initializes the Visualizer server and configures chunkers.

Parameters:

  • host

    (str, default: '127.0.0.1' ) –

    Host IP to run the server. Defaults to "127.0.0.1".

  • port

    (int, default: 8000 ) –

    Port number to run the server. Defaults to 8000.

  • token_counter

    (Callable[[str], int] | None, default: None ) –

    Function to count tokens in text/code. Required for chunkers if used with max_tokens.

Methods:

  • serve –

    Starts the FastAPI server and prints the server URL.

Source code in src/chunklet/visualizer/visualizer.py 
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
@validate_input
def __init__(
    self,
    host: str = "127.0.0.1",
    port: int = 8000,
    token_counter: Callable[[str], int] | None = None,
):
    """Initializes the Visualizer server and configures chunkers.

    Args:
        host: Host IP to run the server. Defaults to "127.0.0.1".
        port: Port number to run the server. Defaults to 8000.
        token_counter: Function to count tokens
            in text/code. Required for chunkers if used with `max_tokens`.
    """
    if FastAPI is None:
        raise ImportError(
            "The 'fastapi' library is not installed. "
            "Please install it with 'pip install fastapi>=0.115.12' or install the visualization extras "
            "with 'pip install 'chunklet-py[viz]''"
        )

    if msgpack is None:
        raise ImportError(
            "The 'msgpack' library is not installed. "
            "Please install it with 'pip install msgpack>=1.0.8' or install the visualization extras "
            "with 'pip install 'chunklet-py[viz]''"
        )

    self.host = host
    self.port = port
    self._token_counter = token_counter

    self.app = FastAPI()

    # Initialize chunkers with sensible defaults; constraint attributes are
    # mutated per request from the submitted params.
    self.document_chunker = DocumentChunker(
        lang="en",
        token_counter=token_counter,
        max_sentences=3,
    )
    self.code_chunker = CodeChunker(token_counter=token_counter, max_lines=50)

    self.static_dir = Path(__file__).parent / "static"

    self.app.mount(
        "/static", StaticFiles(directory=str(self.static_dir)), name="static"
    )

    # API endpoints
    self.app.get("/api/token_counter_status")(self._get_token_counter_status)
    self.app.get("/health")(self._get_health_check)
    self.app.get("/")(self._get_index)
    self.app.post("/api/chunk")(self._chunk_file)

token_counter property writable 

token_counter

Get the current token counter function.

serve 

serve()

Starts the FastAPI server and prints the server URL.

Source code in src/chunklet/visualizer/visualizer.py 
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
def serve(self):
    """Starts the FastAPI server and prints the server URL."""
    if uvicorn is None:
        raise ImportError(
            "The 'uvicorn' library is not installed. "
            "Please install it with 'pip install uvicorn>=0.34.0' or install the visualization extras "
            "with 'pip install 'chunklet-py[viz]''"
        )

    print(" =" * 20)
    print("\nTEXT CHUNK VISUALIZER")
    print("= " * 20)
    print(f"URL: http://{self.host}:{self.port}")

    uvicorn.run(
        self.app,
        host=self.host,
        port=self.port,
        access_log=False,
    )