Older AWS tutorials use the name Amazon Elasticsearch Service because that was the managed search service’s original branding. AWS renamed it Amazon OpenSearch Service on September 8, 2021. The current service supports modern OpenSearch engines and a defined set of legacy open-source Elasticsearch versions; the original name is not a recommendation to start a fresh production application on an outdated engine. This walkthrough keeps the historical term in the page title for readers maintaining those deployments, while using today’s Amazon OpenSearch Service concepts and console paths for new work.
The practical objective is narrow: create an isolated search environment for a fictional support team’s application logs, index a small set of synthetic events, answer a useful query, watch for ingestion and performance errors, then remove or migrate the test resources safely. It is not a recipe for opening an internet-facing production cluster. Search engines hold copied data, often including sensitive identifiers; network boundaries, authorization, mapping choices and retention deserve attention before the first document is written. Readers seeking the broader product background can use the existing Amazon OpenSearch service fundamentals alongside this hands-on sequence.
Understand what changed from Amazon Elasticsearch Service
An Amazon OpenSearch Service domain is a managed OpenSearch cluster with a selected engine, compute, storage, networking and security configuration. You choose an engine version that the service supports; AWS operates the managed service infrastructure but you remain responsible for the application’s index model, account controls, ingest behavior and many performance decisions. The legacy Elasticsearch OSS support list ends with version 7.10, which is not the same as claiming every older distribution or plugin is compatible with a current domain. For a new workload, prefer a supported current OpenSearch engine after checking required plugins and client compatibility.
The rename also changed operational names. OpenSearch Dashboards succeeded the former Kibana naming for the supported engine, and current administration uses the aws opensearch CLI namespace and OpenSearch Service APIs rather than copying an old aws es walkthrough. That doesn’t mean all existing endpoints instantly changed or all old clients stopped working. A legacy deployment may continue to serve traffic on its existing endpoint while it is assessed for an upgrade. Test the exact domain’s engine version, client libraries and API behavior before changing application code.
The distinction matters because a search-service upgrade can alter mappings, query behavior, authentication and index compatibility. A screenshot from an older AWS console is not reliable evidence that a similarly named security option has the same default today. AWS publishes version-specific upgrade paths and support timelines; record the actual source engine and check those records when planning migration. Keep older Elasticsearch-era deployment considerations as historical context, not as blanket current implementation guidance.
Before creating resources, identify why a search index is useful. It is optimized for retrieving and aggregating indexed documents, such as application logs or product descriptions. It is not a substitute for a transactional database that owns authoritative orders or payments. Decide which upstream system owns the facts, how recent search results must be and how the index will recover if it becomes unavailable. That decision prevents coupling business-critical writes to an eventually updated search copy.
Choose a provisioned domain or a serverless collection
Amazon OpenSearch Service offers provisioned domains and OpenSearch Serverless collections. In a provisioned domain, you select instance capacity and storage, then handle more of the scaling, shard placement and operational planning. In Serverless, AWS automatically manages the compute resources for collections; you still need collection-level access, encryption, network policies and controls over ingestion and workload. Serverless does not mean anonymous access, zero capacity cost or identical configuration to a provisioned domain.
For the support-log example, a small temporary provisioned domain makes resource decisions visible: a chosen engine, node type, storage and shard allocation. A serverless collection may be appropriate if automatic scaling and workload isolation matter more than controlling individual domain nodes. Decide based on actual ingestion volume, retention, latency, feature needs and the service’s current pricing rather than the marketing word “serverless.” OpenSearch Serverless has workload-specific supported features and a separate security-policy model; consult the official domain-versus-collection comparison before expecting a feature to work the same way.
Write down a realistic workload before selecting capacity. Suppose the application produces a few thousand events per day in a lab but could eventually produce millions in production. The lab should represent document shape and query patterns, not pretend that its tiny data volume proves production shard sizing. Estimate bytes after indexing and replicas, expected query concurrency, peak ingest rates and any requirements for Availability Zone resilience. It is better to use a controlled test with cost alerts than to guess an instance count from an old blog post.
If your purpose is only to learn query semantics, consider an appropriately authorized existing sandbox or local environment rather than creating an expensive managed resource. Some service configurations incur charges as soon as capacity is provisioned, even when no search requests are made. Record a teardown owner and time, budget alerts and which storage or snapshots may remain afterward. Never assume every option is covered by a free usage tier.
Create the search environment with access controls first
Begin in a dedicated non-production AWS account with an authenticated human or automation role. Confirm the intended account and Region before choosing a domain name. For a provisioned domain, select a supported OpenSearch engine, a capacity and storage profile suitable for the lab, and network placement. A VPC-based domain can help limit reachability to intended networks and clients; it does not automatically authorize every HTTP operation. A public endpoint is not itself an authorization policy and should not be made broadly accessible simply to test a command from a laptop.
Review encryption at rest, node-to-node encryption and HTTPS enforcement. Fine-grained access control can limit operations by roles, indexes, documents or fields, but its interactions with IAM and the domain access policy must be understood. AWS documents a multi-layer model: network reachability, domain access policy and fine-grained permissions. For an internal service, start with a deny-by-default approach and grant only the application actions required. An unrestricted wildcard policy is not an acceptable remedy for an AccessDenied response.
The administrative identity belongs to AWS IAM; the identities and roles used for search API calls may need appropriately signed AWS Signature Version 4 requests. If the domain uses IAM principal authentication, do not mix an unrelated HTTP Basic Authentication example into the same client without reviewing the supported model. When fine-grained access control is enabled, its role mapping also matters. A successfully authenticated principal can still lack index permissions, which is the expected behavior of a correctly restricted system.
Create a distinct application role for writing support events and another role for reading aggregated reports. The writer should not be able to delete unrelated indexes or change domain configuration; the reader should not have write permissions. Test each role with permitted and denied operations. The support dashboard may need aggregated counts but not the full text of personally identifying details. Apply field- or document-level controls where the domain’s configuration supports them and the threat model requires them.
Define a useful index and ingest synthetic documents
An index stores documents for search and aggregation. Before sending data, define which fields are exact identifiers, which are searchable text, which are dates and which are numbers. In the support-event example, service and severity are good exact-match filters, timestamp should support time ranges, and message may need full-text analysis. A free-form message should not be the primary source of a customer identifier if accurate access filtering matters. Consistent document structure makes queries understandable and prevents accidental mapping conflicts.
A managed OpenSearch domain exposes REST APIs for index creation and document operations, but those endpoints must be called through the network and authorization path you configured. In a private VPC deployment, use an authorized client with network connectivity to the domain; do not copy an unauthenticated public curl example and then loosen network policy to make it succeed. For a production application, prefer a supported SDK or client with the appropriate authentication/signing flow.
{
"mappings": {
"properties": {
"timestamp": {"type": "date"},
"service": {"type": "keyword"},
"severity": {"type": "keyword"},
"message": {"type": "text"},
"durationMs": {"type": "integer"}
}
}
}The sample mapping represents the body of an index-creation request, not a complete deployment command. Try a small synthetic collection of events: a service timeout, a normal response and a retried background job. Include timestamps and a stable document ID so retries do not create accidental duplicates. If user-generated fields vary unpredictably, allowing them all to become new indexed fields can create mapping growth and resource pressure. Validate the schema and avoid blindly indexing private payloads or authentication headers.
For log-style data, use an index naming and lifecycle strategy that supports retention and recovery. Many environments use time-based indexes and policies for rollover and deletion, but the appropriate mechanism depends on the OpenSearch engine and whether the workload is provisioned or serverless. A six-month search retention target must account for index storage, replicas, snapshots and any underlying retention rules. Deleting an index is not a substitute for removing data from the authoritative source when compliance requires broader erasure.
Ask questions the search data can actually answer
OpenSearch queries can combine full-text matching, exact filters, ranges and aggregations. For the support team, “How many checkout requests timed out in the last hour?” is an operationally useful question. It requires at least a service identifier, event timestamp, severity or error category, and perhaps a latency field. A loose keyword search over entire JSON strings will give misleading results if field types and time windows are inconsistent. Separate filtering from scoring so the answer reflects the right subset of events.
{
"query": {
"bool": {
"filter": [
{"term": {"service": "checkout"}},
{"range": {"timestamp": {"gte": "now-1h", "lte": "now"}}}
],
"must": [
{"match": {"message": "timeout"}}
]
}
}
}The query is a representative body for the domain search API. It will return meaningful results only if the sample documents, mappings, permissions and timestamps correspond to those fields. A term query is appropriate for exact keyword values; a match query uses text analysis, and its behavior depends on how the field was indexed. Do not assume the same word will match identically when the analyzer or mapping changes. Validate expected documents with small test fixtures before treating a dashboard count as a business fact.
A dashboard can expose search hits, grouped counts and trends, but it should also reveal missing or delayed data. Search analytics is rarely a perfect real-time mirror of the source system. Compare event generation time with ingestion and indexing times, and confirm the query includes the relevant time zone. A sudden zero count could mean a healthy service, a broken ingestion pipeline or a dashboard filter that excludes the events. Operational users need enough context to tell those cases apart.
Search workloads often need aggregation and pagination. A query that returns a handful of examples is not a complete export. Use bounded result sets, an appropriate pagination or point-in-time approach where supported, and carefully scoped aggregations for summary questions. Avoid queries that retrieve sensitive full records when a count or a small set of fields would suffice. The most valuable indexes are those whose access patterns and data contracts are designed from a concrete question.
Connect ingestion without losing or duplicating events
The support application can send individual documents directly to an authorized search endpoint, but that creates coupling between user requests and search availability. Many production designs buffer events through a stream or delivery pipeline so application requests can succeed while search temporarily slows. A pipeline based on Amazon Kinesis streams and managed delivery may be relevant when several downstream systems need the same events or require distinct delivery guarantees. The important decision is the durability and retry contract, not a reflex to insert another AWS service into a diagram.
Define what happens to an event when OpenSearch rejects it because of an invalid mapping, authorization policy, overloaded domain or expired signing credentials. Retrying every error forever can amplify an outage and produce duplicates. A reliable pipeline needs bounded retries for transient failures, explicit handling for permanent schema errors, observable failed-event storage and a replay method. For bulk indexing, examine individual item results rather than assuming an HTTP success proves that every document was indexed.
The index should be rebuildable from a suitable source of truth where business requirements permit. If data originates in a transactional system or durable object storage, maintain a method to reconstruct the search view after accidental deletion or migration. Design stable document identifiers and upsert rules so replay can be idempotent. A search cluster can serve users while still losing the latest events after a network partition; test that situation before promising complete or instantaneous results.
Security review belongs in ingestion as much as in the domain. Avoid copying secrets, authentication tokens and personal data into free-text logs. If the index may contain regulated data, understand retention, redaction and access logging obligations. A change to the application event schema should be versioned and tested against index mappings before a high-volume release; otherwise a single new field or type may cause a burst of rejected documents.
Observe capacity, shards, security and recovery
A healthy domain needs enough storage, CPU, memory and shard capacity for both indexing and queries. AWS’s shard-sizing guidance warns that oversized shards complicate recovery while too many tiny shards waste heap and processing resources. Choose shard count from expected data and node distribution, then measure rather than copying an arbitrary default. Adding replicas can improve search availability but also consumes resources. A green cluster-health indicator does not guarantee that response times are good or that application requests are authorized.
Monitor free storage, cluster status, JVM memory pressure where applicable, indexing/search latency, thread-pool rejections and node behavior using Amazon CloudWatch and OpenSearch’s available diagnostics. Establish a baseline with realistic read and write concurrency. An ingestion spike may reveal a mapping mistake, a backlog or insufficient capacity; a query spike may be caused by an unbounded aggregation or missing filter. Track the application-facing impact, not only cluster metrics.
AWS control-plane changes such as domain configuration updates can be reviewed through the relevant audit mechanisms, including AWS CloudTrail for supported management API events. Search API access and document reads require their own appropriate access and audit strategy; do not assume ordinary CloudTrail management events represent each search query. Separate cluster administration, application ingestion and analytical read access. In incident response, the question “Who changed the network or access policy?” is different from “Which user searched for a particular document?”
Resilience requires more than a snapshot checkbox. Confirm service-supported automated snapshots and the backup/restore approach that matches the domain configuration, as well as how long rebuilding an index would take. Test a recovery into an appropriately isolated environment. For operationally important data, check multi-AZ configuration, index replication, recovery time and recovery point expectations. A lab that has never tested failed nodes, rejected ingestion or restore behavior is not evidence of production readiness.
Plan legacy upgrades and remove test resources cleanly
If an organization still runs an old Amazon Elasticsearch Service domain, begin with inventory: exact engine version, application and client libraries, index mappings, shard sizes, access policies, plugin assumptions, query behaviors and snapshot state. AWS supports selected legacy Elasticsearch OSS versions and publishes per-version standard and extended support schedules, with potential charges after standard support ends. Do not confuse “the service still accepts this engine” with “this engine is recommended for a new deployment.” Account for version-specific upgrade constraints and potential breaking changes before scheduling a change.
A migration may use snapshots and restore, reindexing, dual writing or a staged source-to-target ingestion pipeline. The right method depends on compatibility, acceptable downtime, source of truth and verification needs. Test queries that matter to users, including exact keyword matching, aggregations and security filtering, on a realistic sample before switching traffic. Plan a rollback window, monitor both old and new systems during transition and avoid treating a renamed console label as proof of application compatibility.
When the lesson is finished, stop ingestion first and confirm that no application depends on the test endpoint. Delete the test domain or collection through its supported AWS workflow, then check for residual snapshots, pipelines, logs and security resources that may continue to incur charges. Do not delete a shared VPC component or production IAM role as part of cleaning a personal lab. Keep the final configuration, sample mapping and verified query examples in source control or documentation without storing private data or credentials.
The historical “AWS Elasticsearch” vocabulary still helps readers understand older architectures, but the operational starting point in 2026 is Amazon OpenSearch Service. A useful first deployment is not measured by how quickly a dashboard opens. It is measured by whether the team can explain the data owner, secure endpoint, permitted actions, index mapping, ingestion failure path, search semantics, capacity limits and recovery plan. Those decisions turn a simple log-search experiment into a foundation that can safely evolve with the service.