You can use native Elasticsearch (up to version 7.10) or OpenSearch APIs, such as the index and bulk APIs, to load data into your domain. \n"},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-amazon-opensearch-service-support-integration-with-logstash-6","itemHeading":"Does Amazon OpenSearch Service support integration with Logstash?","itemLongLoc":"
Yes. Amazon OpenSearch Service supports integration with Logstash. You can set up your Amazon OpenSearch Service domain as the backend store for all logs coming through your Logstash implementation. You can set up access control on your Amazon OpenSearch Service domain to either use request signing to authenticate calls from your Logstash implementation, or use resource based IAM policies to include IP addresses of instances running your Logstash implementation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-amazon-opensearch-service-support-integration-with-kibana-7","itemHeading":"Does Amazon OpenSearch Service support integration with Kibana?","itemLongLoc":"
Yes. Amazon OpenSearch Service offers visualization capabilities powered by OpenSearch Dashboards and Kibana (1.5 to 7.10 versions)."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-storage-options-are-available-with-amazon-opensearch-service-8","itemHeading":"What storage options are available with Amazon OpenSearch Service?","itemLongLoc":"
You can choose between local on-instance storage or EBS volumes. During domain creation, if you select EBS storage, you can increase and decrease the size of the storage volume as necessary."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-types-of-ebs-volumes-does-amazon-opensearch-service-support-9","itemHeading":"What types of EBS volumes does Amazon OpenSearch Service support?","itemLongLoc":"
You can choose between Magnetic, General Purpose, and Provisioned IOPS EBS volumes."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#is-there-a-limit-on-the-amount-of-storage-that-can-be-allocated-to-an-amazon-opensearch-service-domain-10","itemHeading":"Is there a limit on the amount of storage that can be allocated to an Amazon OpenSearch Service domain?","itemLongLoc":"
Yes. Amazon OpenSearch Service deploys storage based the choice of instance and/or the size of the associated EBS volume. The maximum storage per node is 24 TB with R6g.12xlarge instances with EBS gp3 storage. With the default maximum of 80 data nodes allowed per Amazon OpenSearch Service domain, you can allocate about 1920 TB of storage to a single domain. You can request a service limit increase up to 200 instances per domain by creating a case with the AWS Support Center. With 200 instances, you can allocate about 3 PB of storage to a single domain."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-are-dedicated-master-instances-distributed-across-azs-11","itemHeading":"How are dedicated master instances distributed across AZs?","itemLongLoc":"
If you deploy your data instances in a single AZ, your dedicated master instances are also deployed in the same AZ. However, if you deploy your data instances across two or three AZs, Amazon OpenSearch Service automatically distributes the dedicated master instances across three AZs. The exception to this rule occurs if a region only has two AZs or if you select an older-generation instance type for the master instances that is not available in all AZs. For more details, refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-the-recommended-az-configuration-for-production-workloads-12","itemHeading":"What is the recommended AZ configuration for production workloads?","itemLongLoc":"
For production workloads, we recommend deploying your data instances across three AZs since it offers better availability. Also, we recommend provisioning instances in multiples of three for equal distribution across AZs. In regions where three AZs are not available, we recommend using a two AZ deployment with an even number of data instances. In all cases, we recommend provisioning three dedicated master instances."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-configure-my-domain-for-three-az-deployment-13","itemHeading":"How can I configure my domain for three AZ deployment?","itemLongLoc":"
You can enable three AZ deployment for both existing and new domains using the AWS console, CLI or SDKs. For more details, refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#is-there-a-fee-for-enabling-three-az-deployment-14","itemHeading":"Is there a fee for enabling three AZ deployment?","itemLongLoc":"
No. Amazon OpenSearch Service does not charge anything for enabling three AZ deployment. You only pay for the number of instances in your domain, not the number of AZs to which they are deployed."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#i-no-longer-see-the-zone-awareness-option-in-my-console-is-my-domain-no-longer-zone-aware-15","itemHeading":"I no longer see the “zone awareness” option in my console. Is my domain no longer zone aware?","itemLongLoc":"
All domains configured for multiple AZs will have zone awareness enabled to ensure shards are distributed across Availability Zones. In the console, you can now explicitly choose two or three AZ deployments. Domains previously configured with “Zone Awareness” will continue to be deployed across two AZs unless they are reconfigured. For more details, refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-amazon-opensearch-service-handle-instance-failures-and-az-disruptions-16","itemHeading":"How does Amazon OpenSearch Service handle instance failures and AZ disruptions?","itemLongLoc":"
If one or more instances in an AZ are unreachable or not functional, Amazon OpenSearch Service automatically tries to bring up new instances in the same AZ to replace the affected instances. In the rare event that new instances cannot be brought up in the AZ, Amazon OpenSearch Service brings up new instances in the other available AZs if the domain has been configured to deploy instances across multiple AZs. Once the AZ issue resolves, Amazon OpenSearch Service rebalances the instances such that they are equally distributed across the AZs configured for the domain. For more details refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#if-i-have-only-one-replica-for-the-indices-in-my-domain-should-i-use-two-or-three-azs-17","itemHeading":"If I have only one replica for the indices in my domain, should I use two or three AZs?","itemLongLoc":"
Even when you configure one replica, we recommend three AZs. If an AZ disruption occurs in a three AZ domain, you only lose one-third of your capacity but if the disruption occurs in a two AZ domain, you lose half your capacity, which can be more disruptive. Also, in a three AZ domain, when an AZ is disrupted, Amazon OpenSearch Service can fall back to the two remaining AZs, and still support cross-AZ replication . In a two AZ domain, you lose cross-AZ replication if one AZ is disrupted, which can further reduce availability. For more details refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-do-i-leverage-three-az-deployment-for-my-vpc-domain-18","itemHeading":"How do I leverage three AZ deployment for my VPC domain?","itemLongLoc":"
The number of AZs your domain is deployed to corresponds to the number of subnets you have configured for your VPC domain. You need to configure at least three subnets in your VPC domain to enable three AZ deployment. For more details on configuring VPC, refer our documentation."},"metadata":{"tags":[{"name":"Setup and configuration","namespaceId":"awt-content-topics#ams#c10","id":"awt-content-topics#ams#c10#setup-and-configuration-1"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-programs-running-on-servers-in-my-own-data-center-access-my-amazon-opensearch-service-domains-0","itemHeading":"Can programs running on servers in my own data center access my Amazon OpenSearch Service domains?","itemLongLoc":"
Yes. The programs with public Internet access can access Amazon OpenSearch Service domains through a public endpoint. If your data center is already connected to Amazon VPC through Direct Connect or SSH tunneling, you can also use VPC access. In both cases, you can configure IAM policies and security groups to allow programs running on servers outside of AWS to access your Amazon OpenSearch Service domains. Click here for more information about signed requests."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-migrate-data-from-my-existing-opensearchelasticsearch-cluster-to-a-new-amazon-opensearch-service-domain-1","itemHeading":"How can I migrate data from my existing OpenSearch/Elasticsearch cluster to a new Amazon OpenSearch Service domain?","itemLongLoc":"
To migrate data from an existing Elasticsearch or OpenSearch cluster, you should create a snapshot of an existing cluster, and store the snapshot in your Amazon S3 bucket. Then you can create a new Amazon OpenSearch Service domain and load data from the snapshot into the newly created Amazon OpenSearch Service domain using the restore API."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-scale-an-amazon-opensearch-service-domain-2","itemHeading":"How can I scale an Amazon OpenSearch Service domain?","itemLongLoc":"
Amazon OpenSearch Service allows you to control the scaling of your Amazon OpenSearch Service domains using the console, API, and CLI. You can scale your Amazon OpenSearch Service domain by adding, removing, or modifying instances or storage volumes depending on your application needs. Amazon OpenSearch Service is integrated with Amazon CloudWatch to provide metrics about the state of your Amazon OpenSearch Service domains to enable you to make appropriate scaling decisions for your domains."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-scaling-my-amazon-opensearch-service-domain-require-downtime-3","itemHeading":"Does scaling my Amazon OpenSearch Service domain require downtime?","itemLongLoc":"
No. Scaling your Amazon OpenSearch Service domain by adding or modifying instances, and storage volumes is an online operation that does not require any downtime."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-amazon-opensearch-service-support-cross-zone-replication-4","itemHeading":"Does Amazon OpenSearch Service support cross-zone replication?","itemLongLoc":"
Yes. If you enable replicas for your OpenSearch/Elasticsearch indices and use multiple Availability Zones, Amazon OpenSearch Service automatically distributes your primary and replica shards across instances in different AZs."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-amazon-opensearch-service-expose-any-performance-metrics-through-amazon-cloudwatch-5","itemHeading":"Does Amazon OpenSearch Service expose any performance metrics through Amazon CloudWatch?","itemLongLoc":"
Yes. Amazon OpenSearch Service exposes several performance metrics through Amazon CloudWatch including number of nodes, cluster health, searchable documents, EBS metrics (if applicable), CPU, memory and disk utilization for data and master nodes. Please refer to the service documentation for a full listing of available CloudWatch metrics."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#i-wish-to-perform-security-analysis-or-operational-troubleshooting-of-my-amazon-opensearch-service-deployment-can-i-get-a-history-of-all-the-amazon-opensearch-service-api-calls-made-on-my-account-6","itemHeading":"I wish to perform security analysis or operational troubleshooting of my Amazon OpenSearch Service deployment. Can I get a history of all the Amazon OpenSearch Service API calls made on my account?","itemLongLoc":"
Yes. AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The AWS API call history produced by AWS CloudTrail enables security analysis, resource change tracking, and compliance auditing. Learn more about AWS CloudTrail at the AWS CloudTrail detail page, and turn it on via CloudTrail's AWS Management Console home page."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-a-snapshot-7","itemHeading":"What is a snapshot?","itemLongLoc":"
A snapshot is a copy of your Amazon OpenSearch Service domain at a moment in time."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#why-would-i-need-snapshots-8","itemHeading":"Why would I need snapshots?","itemLongLoc":"
Creating snapshots can be useful in case of data loss caused by node failure, as well as the unlikely event of a hardware failure. You can use snapshots to recover your Amazon OpenSearch Service domain with preloaded data or to create a new Amazon OpenSearch Service domain with preloaded data. Another common reason to use backups is for archiving purposes. Snapshots are stored in Amazon S3."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-amazon-opensearch-service-provide-automated-snapshots-9","itemHeading":"Does Amazon OpenSearch Service provide automated snapshots?","itemLongLoc":"
Yes. By default, Amazon OpenSearch Service automatically creates hourly snapshots of each Amazon OpenSearch Service domain and retains them for 14 days."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-long-are-the-automated-daily-hourly-snapshots-stored-by-amazon-opensearch-service-10","itemHeading":"How long are the automated daily hourly snapshots stored by Amazon OpenSearch Service?","itemLongLoc":"
Amazon OpenSearch Service will retain the last 14 days worth of automated hourly snapshots."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#is-there-a-charge-for-the-automated-daily-hourly-snapshots-11","itemHeading":"Is there a charge for the automated daily hourly snapshots?","itemLongLoc":"
There is no additional charge for the automated hourly snapshots. The snapshots are stored for free in an Amazon OpenSearch Service S3 bucket and will be made available for node recovery purposes."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-create-additional-snapshots-of-my-amazon-opensearch-service-domains-as-needed-12","itemHeading":"Can I create additional snapshots of my Amazon OpenSearch Service domains as needed?","itemLongLoc":"
Yes. You can use the snapshot API to create additional manual snapshots in addition to the daily-automated snapshots created by Amazon OpenSearch Service. The manual snapshots are stored in your S3 bucket and will incur relevant Amazon S3 usage charges."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-snapshots-created-by-the-manual-snapshot-process-be-used-to-recover-a-domain-in-the-event-of-a-failure-13","itemHeading":"Can snapshots created by the manual snapshot process be used to recover a domain in the event of a failure?","itemLongLoc":"
Yes. Customers can create a new Amazon OpenSearch Service domain and load data from the snapshot into the newly created Amazon OpenSearch Service domain using the OpenSearch/Elasticsearch restore API."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-happens-to-my-snapshots-when-i-delete-my-amazon-opensearch-service-domain-14","itemHeading":"What happens to my snapshots when I delete my Amazon OpenSearch Service domain?","itemLongLoc":"
The daily snapshots retained by Amazon OpenSearch Service will be deleted as part of domain deletion. Before deleting a domain, you should consider creating a snapshot of the domain in your own S3 buckets using the manual snapshot process. The snapshots stored in your S3 bucket will not be affected if you delete your Amazon OpenSearch Service domain."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-types-of-opensearchelasticsearch-logs-are-exposed-by-amazon-opensearch-service-15","itemHeading":"What types of OpenSearch/Elasticsearch logs are exposed by Amazon OpenSearch Service?","itemLongLoc":"
Amazon OpenSearch Service exposes three Elasticsearch or OpenSearch logs through Amazon CloudWatch Logs: error logs, search slow logs, and index slow logs. These logs are useful for troubleshooting performance and stability issues with one’s domain."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-slow-logs-16","itemHeading":"What are slow logs?","itemLongLoc":"
Slow logs are log files that help track the performance of various stages in an operation. OpenSearch and Elasticsearch exposes two kinds of slow logs: \n
\n
Index Slow Logs – These logs provide insights into the indexing process and can be used to fine-tune the index setup. \n
Search Slow Logs – These logs provide insights into how fast or slow queries and fetches are performing. These logs help fine tune the performance of any kind of search operation on OpenSearch or Elasticsearch. \n \n
For complete details on slow logs, please refer to OpenSearch documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-enable-slow-logs-on-amazon-opensearch-service-17","itemHeading":"How can I enable slow logs on Amazon OpenSearch Service?","itemLongLoc":"
Slows logs can be enabled via the click of a button from the Console or via our CLI and APIs. For more details please refer to our documentation ."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-only-enable-slow-logs-for-specific-indices-18","itemHeading":"Can I only enable slow logs for specific indices?","itemLongLoc":"
Yes. You can update the settings for a specific index to enable or disable slow logs for it. For more details refer to our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-turning-on-slow-logs-in-amazon-opensearch-service-automatically-enable-logging-for-all-indexes-19","itemHeading":"Does turning on slow logs in Amazon OpenSearch Service automatically enable logging for all indexes?","itemLongLoc":"
No. Turning on slow logs in Amazon OpenSearch Service enables the option to publish the generated logs to Amazon CloudWatch Logs for indices in the given domain. However, in order to generate the logs you have to update the settings for one or more indices to start the logging process. For more details on setting the index configuration for enabling slow logs, please refer to our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#if-i-turn-off-the-slow-logs-in-amazon-opensearch-service-does-it-mean-that-log-files-are-no-longer-being-generated-20","itemHeading":"If I turn off the slow logs in Amazon OpenSearch Service, does it mean that log files are no longer being generated?","itemLongLoc":"
No. The generation of log files are dependent on the index settings. To turn off generation of the log files you have to update the index configuration. For more details on setting the index configuration for enabling slow logs, see our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-change-the-granularity-of-logging-21","itemHeading":"Can I change the granularity of logging?","itemLongLoc":"
You can only change the granularity of logging for slow logs. OpenSearch and Elasticsearch expose multiple levels of logging for slow logs. You need to set the appropriate level in the configuration of your index. For more details on setting the index configuration for enabling slow logs, please refer to OpenSearch documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-enabling-slow-logs-or-error-logs-cost-me-anything-22","itemHeading":"Will enabling slow logs or error logs cost me anything?","itemLongLoc":"
When slow logs or error logs are enabled, Amazon OpenSearch Service starts publishing the generated logs to CloudWatch Logs. Amazon OpenSearch Service does not charge anything for enabling the logs. However, standard CloudWatch charges will apply."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-kinds-of-error-logs-are-exposed-by-amazon-opensearch-service-23","itemHeading":"What kinds of error logs are exposed by Amazon OpenSearch Service?","itemLongLoc":"
OpenSearch uses Apache Log4j 2 and its built-in log levels (from least to most severe) of TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. If you enable error logs, Amazon OpenSearch Service publishes log lines of WARN, ERROR, and FATAL, and select errors from the DEBUG level to CloudWatch. For more details, refer our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-enable-error-logs-on-amazon-opensearch-service-24","itemHeading":"How can I enable error logs on Amazon OpenSearch Service?","itemLongLoc":"
Error logs can be enabled via the click of a button from the AWS Console or via our CLI and APIs. For more details please refer to our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-enable-error-logs-for-only-specific-indices-25","itemHeading":"Can I enable error logs for only specific indices?","itemLongLoc":"
No, error logs are exposed for the entire domain. That is, once enabled, log entries from all indices in the domain will be made available."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#are-error-logs-available-for-all-versions-of-elasticsearch-supported-by-amazon-opensearch-service-26","itemHeading":"Are error logs available for all versions of Elasticsearch supported by Amazon OpenSearch Service?","itemLongLoc":"
No, error logs are available only for Elasticsearch versions 5.x and above."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#is-there-any-limit-on-the-size-of-each-log-entry-27","itemHeading":"Is there any limit on the size of each log entry?","itemLongLoc":"
Yes. Each log entry made into CloudWatch will be limited to 255,000 characters. If your log entry is bigger than that, it will be truncated to 255,000 characters."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-the-recommended-best-practice-for-using-slow-logs-28","itemHeading":"What is the recommended best practice for using slow logs?","itemLongLoc":"
Slow logs are only needed when you want to troubleshoot your indexes or fine-tune performance. The recommended approach is to only enable logging for those indexes for which you need additional performance insights. Also, once the investigation is done, you should turn off logging so that you don’t incur any additional costs on account of it. For more details, see our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-consume-logs-from-cloudwatch-logs-29","itemHeading":"How can I consume logs from CloudWatch Logs?","itemLongLoc":"
CloudWatch offers multiple ways to consume logs. You can view log data, export it to S3, or process it in real time. To learn more, see the CloudWatch Logs developer guide."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#are-slow-logs-available-for-all-versions-of-opensearch-and-elasticsearch-supported-by-amazon-opensearch-service-30","itemHeading":"Are slow logs available for all versions of OpenSearch and Elasticsearch supported by Amazon OpenSearch Service?","itemLongLoc":"
Yes, slow logs can be enabled for all versions of OpenSearch and Elasticsearch supported by Amazon OpenSearch Service. However, there are slight differences in the way log settings can be specified for each version of Elasticsearch. Please refer to our documentation for more details."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-the-cluster-have-any-down-time-when-logging-is-turned-on-or-off-31","itemHeading":"Will the cluster have any down time when logging is turned on or off?","itemLongLoc":"
No. There will not be any down-time. Every time the log status is updated, we will deploy a new cluster in the background and replace the existing cluster with the new one. This process will not cause any down time. However, since a new cluster is deployed the update to the log status will not be instantaneous."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#which-elasticsearch-and-opensearch-versions-does-the-in-place-upgrade-feature-support-32","itemHeading":"Which Elasticsearch and OpenSearch versions does the in-place upgrade feature support?","itemLongLoc":"
Amazon OpenSearch Service currently supports in-place version upgrade for domains with any OpenSearch version or Elasticsearch versions 5.x and above. The target versions that we support for the upgrade are 5.6, 6.3, 6.4, 6.5, 6.7, 6.8, 7.1, 7.4, 7.7, 7.8, 7.9, and 7.10. For more details refer our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#my-domain-runs-a-version-of-elasticsearch-older-than-5x-how-do-i-upgrade-those-domains-33","itemHeading":"My domain runs a version of Elasticsearch older than 5.x. How do I upgrade those domains?","itemLongLoc":"
Please refer to our documentation for details on migrating from various Elasticsearch versions."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-my-domain-be-offline-while-the-in-place-upgrade-is-in-progress-34","itemHeading":"Will my domain be offline while the in-place upgrade is in progress?","itemLongLoc":"
No. Your domain remains available throughout the upgrade process. However, part of the upgrade process involves relocating shards, which can impact domain performance. We recommend upgrading when the load on your domain is low."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-check-if-my-domains-elasticsearch-version-can-be-upgraded-35","itemHeading":"How can I check if my domain’s Elasticsearch version can be upgraded?","itemLongLoc":"
In-place version upgrade is available only for domains running Elasticsearch 5.x and above. If your domain is of version 5.x or above, you can run the upgrade eligibility check to validate whether your domain can be upgraded to the desired version. Please refer to our documentation to learn more."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-tests-done-by-amazon-opensearch-service-to-validate-my-domains-upgrade-eligibility-36","itemHeading":"What are the tests done by Amazon OpenSearch Service to validate my domains upgrade eligibility?","itemLongLoc":"
For detailed list of the tests we run to validate upgrade eligibility, please refer to our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-update-my-domain-configuration-while-the-version-upgrade-is-in-progress-37","itemHeading":"Can I update my domain configuration while the version upgrade is in progress?","itemLongLoc":"
No. Once the in-place version upgrade has been triggered, you cannot make changes to your domain configuration until the upgrade completes or fails. You can continue reading and writing data while the upgrade is in progress. Also, you can delete the domain, in which case the upgrade is terminated and the domain deleted."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-happens-to-the-automated-system-snapshot-when-the-in-place-version-upgrade-is-in-progress-38","itemHeading":"What happens to the automated system snapshot when the in-place version upgrade is in progress?","itemLongLoc":"
The version upgrade process automatically takes a snapshot of the system and only starts the actual upgrade if the snapshot succeeds. If the upgrade is in progress when the automated snapshot’s start time is reached, the automated snapshot is skipped for that day and continued on the next day."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-amazon-opensearch-service-safeguard-against-issues-that-can-crop-up-during-version-upgrades-39","itemHeading":"How does Amazon OpenSearch Service safeguard against issues that can crop up during version upgrades?","itemLongLoc":"
Amazon OpenSearch Service runs a set of tests before triggering the upgrade to check for known issues that can block the upgrade. If no issues are encountered, the service takes a snapshot of the domain and starts the upgrade process if the snapshot is successful. The upgrade is not triggered if there are any issues encountered with any of the steps."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-happens-if-the-system-encounters-issues-while-performing-the-in-place-version-upgrade-40","itemHeading":"What happens if the system encounters issues while performing the in-place version upgrade?","itemLongLoc":"
If encountered issues are minor and fixable, Amazon OpenSearch Service automatically tries to address them and unblock the upgrade. However, if an issue blocks the upgrade, the service reverts back to the snapshot that was taken before the upgrade and logs the error. For more details on viewing the logs from the upgrade progress, please refer to our documentation."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-view-the-history-of-upgrades-on-my-domain-41","itemHeading":"Can I view the history of upgrades on my domain?","itemLongLoc":"
Yes. You can view the upgrade logs from the AWS console or request them using the CLI or SDKs. Please refer to our documentation for more details."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-pause-or-cancel-the-version-upgrade-after-it-has-been-triggered-42","itemHeading":"Can I pause or cancel the version upgrade after it has been triggered?","itemLongLoc":"
No. After the upgrade has been triggered, it cannot be paused or cancelled until it either completes or fails."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-run-in-place-version-upgrade-on-multiple-domains-in-parallel-43","itemHeading":"Can I run in-place version upgrade on multiple domains in parallel?","itemLongLoc":"
Yes. However, if you want to keep all of your domains on the same version, we recommend running the upgrade eligibility check on all domains before upgrading them. This extra step can help catch issues with one domain that might not be present on others."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-long-does-the-in-place-version-upgrade-take-44","itemHeading":"How long does the in-place version upgrade take?","itemLongLoc":"
Depending on the amount of data and the size of the cluster, upgrades can take anywhere from a few minutes to a few hours to complete."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-just-upgrade-the-domain-quickly-without-retaining-any-of-the-data-45","itemHeading":"Can I just upgrade the domain quickly without retaining any of the data?","itemLongLoc":"
No. With in-place version upgrade, all the data in your cluster is also restored as part of the upgrade process. If you only wish to upgrade the domain alone, you can take a snapshot of your data, delete all your indexes from the domain and then trigger an in-place version upgrade. Alternatively, you can create a separate domain with the newer version and then restore your data to that domain."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-downgrade-to-previous-version-if-im-not-comfortable-with-the-new-version-46","itemHeading":"Can I downgrade to previous version if I’m not comfortable with the new version?","itemLongLoc":"
No. If you need to downgrade to an older version, contact AWS Support to restore the automatic, pre-upgrade snapshot on a new domain. If you took a manual snapshot of the original domain, you can perform this step yourself."},"metadata":{"tags":[{"name":"Administration","namespaceId":"awt-content-topics#ams#c29","id":"awt-content-topics#ams#c29#administration-2"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-multi-az-with-standby-0","itemHeading":"What is Multi-AZ with Standby?","itemLongLoc":"
Multi-AZ with Standby is a new deployment option for Amazon OpenSearch Service that enables high-availability and consistent performance for business-critical workloads. With Multi-AZ with Standby, OpenSearch Service managed clusters are resilient to infrastructure failures like node drops, or a single Availability Zone failure, assuring no impact to performance or availability, even in the event of a single Availability Zone failure. Multi-AZ with Standby provides the added benefit of simplifying cluster configuration and management by enforcing best practices and reducing complexity."},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-pre-requisites-for-creating-or-updating-a-cluster-with-multi-az-with-standby-1","itemHeading":"What are the pre-requisites for creating or updating a cluster with Multi-AZ with Standby?","itemLongLoc":"
To enable Multi-AZ with Standby, managed clusters need to meet the following conditions: \n
\n
Run OpenSearch 1.3 or more recent version. \n
Deploy in AWS Regions with 3-AZ. Currently, AWS North California region does not support 3-AZ and are therefore not suitable for Multi-AZ with Standby. \n
Number of data nodes needs to be in multiples of three. \n
Number of data copies (primary + replica) should be in multiples of three. \n
Follow sizing guidelines for the leader (recommended size based on number of nodes, number of shards, and number of mappings in your cluster). \n"},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-failure-scenarios-covered-and-not-covered-by-the-feature-2","itemHeading":"What are the failure scenarios covered and not covered by the feature?","itemLongLoc":"
With Multi-AZ with Standby, Amazon OpenSearch Service detects and automatically recovers from some of the infrastructure failures. Amazon OpenSearch Service automatically fails over from active to standby nodes in under a minute when any of the following events occur: \n
\n
Loss of one active AZ or all nodes in an active AZ \n
Loss of connectivity to one active AZ \n
Instance hardware failure in the active AZ \n
Storage failure on a node in the active AZ \n \n
Currently Multi-AZ with Standby does not cover the following events: \n
\n
Loss of master Quorum, as recovery from this event can take several minutes \n
Loss of multiple Availability Zones \n
Loss of connectivity to a Region \n
Loss of more than 50% nodes in more than one AZ \n
Downtime caused due to insufficient compute or storage as a result of change in workload characteristics \n
Downtime caused due to rouge queries \n
Loss of one or more services that Amazon OpenSearch Service depends on like ARPS and ALB \n
Downtime of OpenSearch Dashboard during version upgrades \n"},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#do-managed-clusters-that-use-multi-az-with-standby-need-to-be-sized-differently-how-do-we-size-managed-clusters-for-multi-az-with-standby-3","itemHeading":"Do managed clusters that use Multi-AZ with Standby need to be sized differently? How do we size managed clusters for Multi-AZ with Standby?","itemLongLoc":"
No. In principle, the sizing guidelines remain the same. Multi-AZ with Standby has pre-requisites that simplify the mental model required to size the cluster. The way to think of sizing for managed cluster is that you should look at the capacity needed to service your workload and then add 50% for redundancy. The primary difference between the current ‘Zone Awareness’ option and the Multi-AZ with Standby option is how redundant or additional capacity is handled to maintain availability. Multi-AZ with Standby requires you have at least one copy of data in each AZ, so that Multi-AZ with Standby can explicitly reserve capacity in one AZ as standby. This standby capacity acts as a failover target during AZ disruption or instance failure. The existing model requires that you maintain an optimum levels of resources to serve your workload. You would continue to monitor your cluster for sizing issues and take corrective actions as the workload characteristics changes."},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-choosing-multi-az-with-standby-mean-i-no-longer-have-to-ensure-my-cluster-is-properly-sized-and-resourced-for-my-workload-4","itemHeading":"Does choosing Multi-AZ with Standby mean I no longer have to ensure my cluster is properly sized and resourced for my workload?","itemLongLoc":"
No. Amazon OpenSearch Service works on a shared responsibility model. You are responsible for ensuring that your cluster is sized in accordance with your workload. Multi-AZ with Standby make the mental model of setting up your cluster simple. You should continue to monitor the error and latency metrics along with storage, CPU and RAM utilization for signals that the cluster is overloaded and may need to be scaled."},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-i-incur-additional-cost-if-i-use-multi-az-with-standby-5","itemHeading":"Will I incur additional cost if I use Multi-AZ with Standby?","itemLongLoc":"
No. The feature Multi-AZ with Standby is available at no extra cost. You continue to pay for the resources deployed on the cluster to serve your workload. If your cluster already follows the best practices and has at least three copies of data for a 3-AZ cluster, you are unlikely to incur additional cost in moving to Multi-AZ with Standby. However, if your cluster is undersized or does not have enough redundant capacity to serve your workload, you will need to add capacity to move to Multi-AZ with Standby for enhanced availability and performance. The standby capacity is reserved from the configured total capacity."},"metadata":{"tags":[{"name":"Multi-AZ with Standby","namespaceId":"awt-content-topics#ams#c76","id":"awt-content-topics#ams#c76#multi-az-with-standby-3"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-does-the-amazon-opensearch-service-sla-guarantee-0","itemHeading":"What does the Amazon OpenSearch Service SLA guarantee?","itemLongLoc":"
Our Amazon OpenSearch Service SLA guarantees a Monthly Uptime Percentage of at least 99.9% for Amazon OpenSearch Service."},"metadata":{"tags":[{"name":"Service Level Agreement","namespaceId":"awt-content-topics#ams#c82","id":"awt-content-topics#ams#c82#service-level-agreement-4"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-do-i-know-if-i-qualify-for-a-sla-service-credit-1","itemHeading":"How do I know if I qualify for a SLA Service Credit?","itemLongLoc":"
You are eligible for a SLA credit for Amazon OpenSearch Service under the Amazon OpenSearch Service SLA if multi-AZ domains on Amazon OpenSearch Service have a Monthly Uptime Percentage of less than 99.9% during any monthly billing cycle. \n
For full details on all of the terms and conditions of the SLA, as well as details on how to submit a claim, please see the Amazon OpenSearch Service SLA details page."},"metadata":{"tags":[{"name":"Service Level Agreement","namespaceId":"awt-content-topics#ams#c82","id":"awt-content-topics#ams#c82#service-level-agreement-4"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-cross-cluster-search-0","itemHeading":"What is cross-cluster search?","itemLongLoc":"
Cross-cluster search is an Elasticsearch and OpenSearch feature that enables performing queries and aggregation across two connected clusters. It works by setting up a lightweight unidirectional connection between participating clusters."},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-minimum-requirements-for-a-domain-to-participate-in-cross-cluster-search-1","itemHeading":"What are the minimum requirements for a domain to participate in cross-cluster search?","itemLongLoc":"
Domains participating in a cross-cluster search needs to meet the following criteria: \n
\n
Participating domains should be on OpenSearch or Elasticsearch version 6.8 and above \n
Participating domains need to have encryption in transit enabled \n
Participating domains need to have Fine Grained Access Control (FGAC) enabled \n
Participating domains versions should adhere to the same rules as rolling version upgrade \n"},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-instance-types-that-support-cross-cluster-search-2","itemHeading":"What are the instance types that support cross-cluster search?","itemLongLoc":"
Cross-cluster search is currently supported on the following instance types \n
\n
i2, i3 family \n
r3, r4, r5 family \n
m4, m5 family \n
c4, c5, family \n
Graviton family \n"},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-instance-types-that-do-not-support-cross-cluster-search-3","itemHeading":"What are the instance types that do not support cross-cluster search?","itemLongLoc":"
Cross-cluster search is not supported on the t2 and m3 family instances due to technical limitation."},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-domains-in-two-different-aws-accounts-participate-in-cross-cluster-search-4","itemHeading":"Can domains in two different AWS accounts participate in cross-cluster search?","itemLongLoc":"
Yes. Participating domains can belong to two different AWS accounts."},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-domains-in-two-different-aws-regions-participate-in-cross-cluster-search-5","itemHeading":"Can domains in two different AWS regions participate in cross-cluster search?","itemLongLoc":"
Yes."},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-start-using-cross-cluster-search-6","itemHeading":"How can I start using cross-cluster search?","itemLongLoc":"
To get started with cross-cluster search, follow the documentation here."},"metadata":{"tags":[{"name":"Cross-cluster search","namespaceId":"awt-content-topics#ams#c84","id":"awt-content-topics#ams#c84#cross-cluster-search-5"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-cross-cluster-replication-0","itemHeading":"What is cross-cluster replication?","itemLongLoc":"
Cross-cluster replication, a new capability that allows Amazon OpenSearch Service customers to automate copying and synchronizing indices from one cluster to another at low latency in same or different AWS Regions."},"metadata":{"tags":[{"name":"Cross-cluster replication","namespaceId":"awt-content-topics#ams#c91","id":"awt-content-topics#ams#c91#cross-cluster-replication-6"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-minimum-requirements-for-a-domain-to-participate-in-cross-cluster-replication-1","itemHeading":"What are the minimum requirements for a domain to participate in cross-cluster replication?","itemLongLoc":"
Domains participating in a cross-cluster replications needs to meet the following criteria: \n
\n
Participating domains should be on Elasticsearch version 7.10 \n
Participating domains need to have encryption in transit enabled \n
Participating domains need to have Fine Grained Access Control (FGAC) enabled \n
Participating domains versions should adhere to the same rules as rolling version upgrade \n"},"metadata":{"tags":[{"name":"Cross-cluster replication","namespaceId":"awt-content-topics#ams#c91","id":"awt-content-topics#ams#c91#cross-cluster-replication-6"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-domains-in-two-different-aws-regions-participate-in-cross-cluster-replication-2","itemHeading":"Can domains in two different AWS Regions participate in cross-cluster replication?","itemLongLoc":"
Yes. domains in two different AWS Regions can participate in cross-cluster replication."},"metadata":{"tags":[{"name":"Cross-cluster replication","namespaceId":"awt-content-topics#ams#c91","id":"awt-content-topics#ams#c91#cross-cluster-replication-6"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#does-cross-cluster-replication-support-ultrawarm-and-cold-storage-3","itemHeading":"Does cross-cluster replication support Ultrawarm and Cold Storage?","itemLongLoc":"
No. Current implementation of cross-cluster replication does not support Ultrawarm or Cold Storage."},"metadata":{"tags":[{"name":"Cross-cluster replication","namespaceId":"awt-content-topics#ams#c91","id":"awt-content-topics#ams#c91#cross-cluster-replication-6"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-charges-for-cross-cluster-replication-4","itemHeading":"What are the charges for cross-cluster replication?","itemLongLoc":"
Yes. You need to pay standard AWS data transfer charges for the data transferred in and out of Amazon OpenSearch Service."},"metadata":{"tags":[{"name":"Cross-cluster replication","namespaceId":"awt-content-topics#ams#c91","id":"awt-content-topics#ams#c91#cross-cluster-replication-6"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#why-did-the-name-change-to-amazon-opensearch-service-from-amazon-elasticsearch-service-0","itemHeading":"Why did the name change to Amazon OpenSearch Service from Amazon Elasticsearch Service?","itemLongLoc":"
We announced the OpenSearch project, a community-driven, open source fork of Elasticsearch and Kibana, on April 12, 2021. We committed to making a long-term investment in OpenSearch to ensure users continue to have a secure, high-quality, fully open source search and analytics suite with a rich roadmap of new and innovative functionality. This project includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2). We launched version 1.0 of OpenSearch on July 12, 2021. As part of our long-term commitment to OpenSearch, we added support for OpenSearch 1.0 on the managed service on September 7, 2021 and changed the name from Amazon Elasticsearch Service to Amazon OpenSearch Service. Along with OpenSearch 1.0, we continue to support legacy Elasticsearch versions until 7.10 on the service. Aside from the name change, you can rest assured that we will continue to deliver the same great experience without any impact to ongoing operations, development methodology, or business use. Learn more about OpenSearch here: https://opensearch.org"},"metadata":{"tags":[{"name":"Name change","namespaceId":"awt-content-topics#ams#c96","id":"awt-content-topics#ams#c96#name-change-7"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#do-i-as-a-customer-have-to-take-any-action-as-part-of-this-name-change-1","itemHeading":"Do I, as a customer, have to take any action as part of this name change?","itemLongLoc":"
We have strived to make this name change as seamless as possible for you. There are aspects, such as the new SDK/configuration APIs, that require your action to ensure you derive the best benefits from the service. While the existing SDK will continue to work from a compatibility perspective, any new functionality that requires new configuration APIs will only be implemented in the new SDK. Hence, we recommend that you move to the new SDK. In addition, irrespective of the new SDK, we strongly recommend that you move your existing IAM policies to use the renamed configuration APIs. As of now, your existing IAM policies will continue to work with the old API definition. However, we will move over to the new API-based permission validation and we will eventually require you to use the new APIs in your policies (specifically for the APIs where there is a name change; e.g. CreateElasticsearchDomain to CreateDomain). Please see the documentation for more details."},"metadata":{"tags":[{"name":"Name change","namespaceId":"awt-content-topics#ams#c96","id":"awt-content-topics#ams#c96#name-change-7"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#do-i-have-to-move-to-the-new-sdk-to-upgrade-to-opensearch-10-2","itemHeading":"Do I have to move to the new SDK to upgrade to OpenSearch 1.0?","itemLongLoc":"
No. From a backward compatibility perspective, we will ensure that your existing setup continues to work with OpenSearch 1.0. However, we recommend that you eventually move to the latest SDK for a cleaner and up-to-date experience, as mentioned above."},"metadata":{"tags":[{"name":"Name change","namespaceId":"awt-content-topics#ams#c96","id":"awt-content-topics#ams#c96#name-change-7"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#are-there-any-changes-to-pricing-with-this-name-change-3","itemHeading":"Are there any changes to pricing with this name change?","itemLongLoc":"
No. There are no changes to pricing. \n
OpenSearch includes certain Apache-licensed Elasticsearch code from Elasticsearch B.V. and other source code. Elasticsearch B.V. is not the source of that other source code. ELASTICSEARCH is a registered trademark of Elasticsearch B.V."},"metadata":{"tags":[{"name":"Name change","namespaceId":"awt-content-topics#ams#c96","id":"awt-content-topics#ams#c96#name-change-7"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#im-using-the-elasticsearch-engine-in-amazon-opensearch-service-why-should-i-upgrade-to-the-opensearch-1x-engine-what-are-the-benefits-for-me-0","itemHeading":"I'm using the Elasticsearch engine in Amazon OpenSearch Service. Why should I upgrade to the OpenSearch 1.x engine? What are the benefits for me?","itemLongLoc":"
Upgrading to OpenSearch 1.x ensures your search infrastructure is built on a growing and dynamic Apache-Licensed open source project and gives you access to a wealth of innovative improvements and features available in OpenSearch 1.2 (as of this writing). Features such as enterprise-grade security, alerting, data-lifecycle management, observability, ML-based anomaly detection, and more are all part of OpenSearch Service, with no additional licensing fees."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-ihave-downtime-if-i-upgrade-1","itemHeading":"Will I have downtime if I upgrade?","itemLongLoc":"
We use a blue/green (BG) deployment process during upgrade. During a BG, the service adds nodes to the OpenSearch Service cluster in the new configuration and version, migrates data from the old nodes, and drops the old nodes when the data migration is complete. During a BG, search and indexing APIs are available and function normally. Although BG is designed not to interfere with query and indexing requests, some changes (especially those involving changes to security-related settings) can cause dashboards to be unavailable during the period of change."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-versions-of-opensearch-and-elasticsearch-does-opensearch-service-support-2","itemHeading":"What versions of OpenSearch and Elasticsearch does OpenSearch Service support?","itemLongLoc":"
OpenSearch supports several versions of Elasticsearch and OpenSearch, some of which already have end of Standard and Extended Support dates announced. For a complete list of engine versions, and corresponding support dates, please see the documentation. For more information about Extended Support, please see the Extended Support. "},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#isaws-deprecating-older-versions-of-elasticsearch-service-3","itemHeading":"Is AWS deprecating older versions of Elasticsearch Service?","itemLongLoc":"
AWS maintains 19 versions of Apache-2.0-licensed Elasticsearch. None of these versions are deprecated or planned for deprecation at this time."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-the-upgrade-trigger-a-bg-if-not-what-is-the-process-for-upgrading-our-nodes-4","itemHeading":"Will the upgrade trigger a BG? If not, what is the process for upgrading our nodes?","itemLongLoc":"
Yes, the upgrade will trigger a BG deployment process. Please review the upgrade preparation and steps here."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#iwant-to-move-to-amazon-opensearch-service-1x-to-take-advantage-of-aws-graviton2-instances-but-i-am-locked-in-with-my-existing-reserved-instances-ris-how-can-you-help-5","itemHeading":"I want to move to Amazon OpenSearch Service 1.x to take advantage of AWS Graviton2 instances, but I am locked in with my existing reserved instances (RIs). How can you help?","itemLongLoc":"
Please work with your AWS account team for information based on your specific situation with RIs."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-should-i-plan-for-before-initiating-an-upgrade-to-amazon-opensearch-service-1x-or-greater-6","itemHeading":"What should I plan for before initiating an upgrade to Amazon OpenSearch Service 1.x or greater?","itemLongLoc":"
The OpenSearch project 1.0 is a fork of open source Elasticsearch 7.10.2. It is wire-compatible with Elasticsearch 7.10--you don’t need to change your usage. To migrate, you can upgrade your domain to version Elasticsearch 7.10 from any prior version in the 6.x and 7.x series, take a snapshot, and restore that snapshot to a domain running OpenSearch Service 1.x. Some clients or tools include version checks that may cause the client or tool to not work with OpenSearch Service. When you upgrade, enable compatibility mode to work around these version checks."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-i-continue-to-use-my-existing-clients-data-collection-and-data-ingestion-tools-withamazon-opensearch-service-1x-7","itemHeading":"Can I continue to use my existing clients, data collection, and data ingestion tools with Amazon OpenSearch Service 1.x?","itemLongLoc":"
In most cases, you can continue to use your existing clients. The APIs and core search functionality are compatible with Elasticsearch version 7.10.2. If you have older clients, the clients perform version checking, or the clients leverage functionality that is targeted to older versions of Elasticsearch such as major versions 5 or 6, then we suggest that you try to bring your clients up to a minimum standard of support on 7.10.2 to ensure a smooth transition. \n
The OpenSearch project supports a broad range of clients that are specifically built to address the OpenSearch versions of the engine on Amazon OpenSearch Service. Check your client against the list of the latest OpenSearch clients and supported programming languages for those clients. \n
You can enable the compatibility mode feature to interoperate with clients from other vendors, but remember to check the version that OpenSearch reports. Enable this setting to ensure the service responds with version 7.10.2 to clients developed prior to the introduction of the OpenSearch Service engine."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#im-running-elasticsearch-version-5x-or-earlier-whats-my-best-upgrade-path-8","itemHeading":"I'm running Elasticsearch version 5.x or earlier. What’s my best upgrade path?","itemLongLoc":"
Elasticsearch 5.x indices are not compatible with Elasticsearch 7.10 or OpenSearch 1.x. You must create a new index and load data from your source. If you are running a log analytics workload, you can evaluate whether your data retention strategy supports running in parallel while you build up a full data set on the new domain."},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#are-there-any-partners-that-can-help-me-with-my-upgrade-9","itemHeading":"Are there any partners that can help me with my upgrade?","itemLongLoc":"
Yes, please contact [email protected] to request a list of partners for your region, industry, and project complexity. AWS Partner Network (APN) partners are trained and have the experience to assist you with upgrades. "},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#will-amazon-opensearch-service-remain-compatible-with-elasticsearch-in-the-future-whats-the-plan-for-the-future-10","itemHeading":"Will Amazon OpenSearch Service remain compatible with Elasticsearch in the future? What’s the plan for the future?","itemLongLoc":"
OpenSearch 1.0 is a fork of Elasticsearch 7.10.2. OpenSearch and Elasticsearch are compatible. If you enable compatibility mode, Elasticsearch clients are also compatible with OpenSearch 1.0. \n
Amazon OpenSearch Service does not and will not offer versions of the Elasticsearch engine after version 7.10.2. \n
As AWS announced when we forked Elasticsearch, we intended to build, and have built a thriving community around OpenSearch. We have published a roadmap for OpenSearch, complete with community input and consensus on the feature priorities. We will make every reasonable effort to maintain compatibility with Elasticsearch. Our goal is to grow with our community and Amazon OpenSearch Service customers. \n
You can directly upgrade to OpenSearch Service 1.0 from Elasticsearch and Kibana versions 6.8.0 to 7.10.2, and open distro for Elasticsearch (ODFE) 1.x. For rolling upgrades from ODFE to OpenSearch, we recommend first upgrading to ODFE 1.13, then upgrading to OpenSearch 1.0. \n
Migration resources are here: \n
Analytics Migrations \n
Migrating to Amazon OpenSearch Service"},"metadata":{"tags":[{"name":"Upgrades","namespaceId":"awt-content-topics#ams#c100","id":"awt-content-topics#ams#c100#upgrades-8"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-is-standard-support-and-extended-support-0","itemHeading":"What is Standard Support and Extended Support?","itemLongLoc":"
Every engine version that is launched in OpenSearch Service is by default covered by Standard Support. As part of Standard Support, AWS provides regular bug fixes and security updates. When Standard Support ends for a version, AWS provides an Extended Support period of at least 12 months after the Standard Support end date. During the Extended Support period, AWS will provide critical security fixes and operating system patches. This gives you more time to plan your upgrade to a more recent supported engine version. When you are running a version that is in Extended Support, you will be charged a flat fee/Normalized instance hour (NIH), in addition to the standard instance and storage costs. Please see the documentation for more information on Extended Support, and the schedule for various versions. For pricing information, please see the pricing page."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#do-we-need-to-opt-in-for-extended-support-1","itemHeading":"Do we need to opt-in for Extended Support?","itemLongLoc":"
No. Domains that are running versions that have reached the end of Standard Support will automatically be covered under Extended Support, and will be priced accordingly. Once you upgrade your domain to a newer version under Standard Support, you will stop being billed for Extended Support."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-are-the-charges-for-extended-support-and-when-do-they-start-2","itemHeading":"What are the charges for Extended Support and when do they start?","itemLongLoc":"
Domains that are running Extended Support will be charged an additional flat fee/Normalized instance hour (NIH), in addition to the standard instance and storage pricing. Please see pricing page for the exact pricing by region. Your domain will be charged for Extended Support automatically from the next day after end of Standard Support. If your domain is running a version that has Standard and Extended Support dates published (see here for details), we will send you a notification on your Personal Health Dashboard, on the OpenSearch Service console, and through EventBridge events, 3-months prior to the end of Standard Support date. For more information on monitoring notifications in OpenSearch Service, please see the documentation here."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-do-i-calculate-extended-support-charges-for-my-domains-3","itemHeading":"How do I calculate Extended Support charges for my domains?","itemLongLoc":"
Domains running versions under Extended Support will be charged an additional flat fee/Normalized Instance Hour (NIH). NIH is computed as a factor of the instance size (e.g. medium, large), and the number of instance hours. For example, if you are running an m7g.medium.search instance for 24 hours in the US East (North Virginia), which is priced at $0.068/hour (on-demand), you will typically pay $1.632 ($0.068x24) as the instance cost. If you are running a version that is in Extended Support, you will pay an additional $0.0065/NIH, which is computed as $0.0065 x 24 (number of instance hours) x 2 (size normalization factor; 2 for medium-sized instances), which comes to $0.312 for Extended Support for 24 hours. The total amount you will pay for 24 hours will be a sum of the standard instance usage cost (excluding storage) and the Extended Support cost, which is $1.944 ($1.632+$0.312). Please see documentation for more information."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-do-i-avoid-extended-support-charges-4","itemHeading":"How do I avoid Extended Support charges?","itemLongLoc":"
You can upgrade your domain to an engine version that is covered by Standard Support. A version is covered under Standard Support until the published end of Standard Support date, or if no end of Standard Support dates have been announced for the version."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#can-we-extend-support-further-beyond-the-extended-support-end-date-5","itemHeading":"Can we extend support further beyond the Extended Support end date?","itemLongLoc":"
No. We recommend that you upgrade to a version that is covered by standard or Extended Support or to a version for which end of support has not been announced. Once Extended Support ends for a version, domains running the specific version will not receive bug fixes or security updates."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-happens-after-the-end-of-extended-support-period-6","itemHeading":"What happens after the end of Extended Support period?","itemLongLoc":"
Once Extended Support ends for a version, we will no longer release bug fixes or security updates for that specific version. Domains running unsupported versions will be isolated and data preserved for at least 30 days. Isolated domains can no longer be accessed, and you will have to restore data into a new domain running a supported version. We strongly recommend that you update your domain to a supported version before the end of Extended Support for a specific version. If you need further assistance, please contact AWS Support."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#are-there-any-restrictions-when-a-domain-is-running-a-version-on-extended-support-for-example-can-i-provision-new-instances-or-create-new-domains-7","itemHeading":"Are there any restrictions when a domain is running a version on Extended Support? For example, can I provision new instances or create new domains?","itemLongLoc":"
Yes. As long the engine version that you wish to use is covered by Extended Support, you can continue to operate the service as usual, without any restrictions."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-fixes-are-included-in-extended-support-8","itemHeading":"What fixes are included in Extended Support?","itemLongLoc":"
As part of Extended Support, AWS provides critical security fixes, and operating system patches as required."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#do-you-support-in-place-upgrades-to-new-versions-without-any-downtime-9","itemHeading":"Do you support in-place upgrades to new versions without any downtime?","itemLongLoc":"
Yes, depending the version you are upgrading from and upgrading to. Please see the documentation here for a list of supported upgrade paths. When you are moving from legacy versions like ES 1.5 or ES 2.3, we do not support in-place upgrades. Please refer to the documentation here for instructions on upgrading your domains running legacy versions."},"metadata":{"tags":[{"name":"Extended Support","namespaceId":"awt-content-topics#ams#c111","id":"awt-content-topics#ams#c111#extended-support-9"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#why-should-i-use-the-zero-etl-integration-of-amazon-opensearch-service-with-amazon-dynamodb-0","itemHeading":"Why should I use the zero-ETL integration of Amazon OpenSearch Service with Amazon DynamoDB?","itemLongLoc":"
This zero-ETL integration with Amazon DynamoDB abstracts away the operational complexity in orchestrating the replication of data from an operational datastore to a search datastore. Data pipelines that are used to keep different datastores in sync can be challenging and costly to build and manage, and suffer from intermittent errors which are difficult to track. This integration enables Amazon DynamoDB customers to obtain near real-time search results from their transactional data by offering a fully managed solution for making operational data from Amazon DynamoDB available in Amazon OpenSearch Service within seconds of being written."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-this-zero-etl-integration-replicate-data-from-amazon-dynamodb-to-amazon-opensearch-service-1","itemHeading":"How does this zero-ETL integration replicate data from Amazon DynamoDB to Amazon OpenSearch Service?","itemLongLoc":"
The zero-ETL integration of Amazon OpenSearch Service with Amazon DynamoDB uses Amazon OpenSearch Ingestion to seamlessly move operational data from Amazon DynamoDB to Amazon OpenSearch Service. To enable an integration, customers first choose the Amazon DynamoDB table whose data needs to be replicated. The zero-ETL integration feature sets up an Amazon OpenSearch Ingestion pipeline in the customer’s account that takes care of replicating the data to an Amazon OpenSearch Service managed cluster or serverless collection. Amazon OpenSearch Ingestion understands the structure of the Amazon DynamoDB tables and then bootstraps an Amazon OpenSearch Service managed cluster or serverless collection with the existing data from the DynamoDB tables. Optionally, customers can specify a schema for the indices that will be created in Amazon OpenSearch Service. Any updates to the DynamoDB table are also replicated to Amazon OpenSearch Service without any manual intervention by customers."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-data-transformation-work-from-while-moving-data-from-amazon-dynamodb-to-amazon-opensearch-service-2","itemHeading":"How does data transformation work from while moving data from Amazon DynamoDB to Amazon OpenSearch Service?","itemLongLoc":"
This zero-ETL features uses Amazon OpenSearch Ingestion to move the data from Amazon DynamoDB to Amazon OpenSearch Service and leverages the native data transformational capabilities of Amazon OpenSearch Ingestion pipelines to aggregate and filter the data while it is in motion. When moving the data from an Amazon DynamoDB table, customers may want to drop a few fields or create new fields based on aggregations across existing fields. Optionally, customers can also write custom logic for Amazon OpenSearch Ingestion to achieve bespoke transformational capability. For other users, who just want to move their entire data from source to sink, Amazon OpenSearch Ingestion provides out-of-the box blueprints so that they can perform the integrations with just a few button clicks."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-security-permissions-are-required-for-using-the-zero-etl-integration-for-dynamodb-3","itemHeading":"What security permissions are required for using the zero-ETL integration for DynamoDB?","itemLongLoc":"
In order to ensure that Amazon OpenSearch Ingestion has the necessary permissions to replicate data across both these systems, the zero-ETL integration feature creates an IAM role with the necessary permissions to read data from Amazon DynamoDB tables and write to an Amazon OpenSearch domain or collection. This role is then assumed by Amazon OpenSearch Ingestion pipelines to ensure that the right security posture is always maintained when moving the data from source to destination."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-monitor-the-state-of-my-integration-between-amazon-dynamodb-and-amazon-opensearch-service-4","itemHeading":"How can I monitor the state of my integration between Amazon DynamoDB and Amazon OpenSearch Service?","itemLongLoc":"
You can view all the metrics related to your zero-ETL integration with Amazon DynamoDB on the dashboards provided by Amazon OpenSearch Ingestion along with real-time logs in Amazon CloudWatch. This enables customers to set up custom alerting that is triggered when user-defined thresholds are breached."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-amazon-opensearch-service-zero-etl-integration-with-amazon-s3-work-5","itemHeading":"How does Amazon OpenSearch Service zero-ETL integration with Amazon S3 work?","itemLongLoc":"
The OpenSearch Service query engine has been rearchitected to support analysis of operational data stored in cloud object stores, such as Amazon S3 and S3-based data lakes. It does so without duplicating data. When seconds make a difference, customers can boost the performance of their queries as well as build fast-loading dashboards by using the new integration’s built-in query acceleration capabilities."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-can-i-get-started-using-amazon-opensearch-zero-etl-integration-with-amazon-s3-6","itemHeading":"How can I get started using Amazon OpenSearch zero-ETL integration with Amazon S3?","itemLongLoc":"
To get started from the AWS Management Console, customers set up a new data source from an existing OpenSearch Service domain running OpenSearch Service version 2.11 or greater. When setting up the new Direct Query data source, customers will need to provide read/write access to Amazon S3 and AWS Glue Data Catalog to facilitate querying data in Amazon S3 from OpenSearch Service. Customers can customize IAM policies to restrict access to specific buckets in Amazon S3 or resources in AWS Glue Data Catalog. After configuring the new data source in the console, customers will go to OpenSearch Service to configure role-based access controls, accelerations to speed up query performance, and optionally out-of-the-box dashboards for popular log type templates such as VPC Flow Logs, Elastic Load Balancer Logs, and NGINX Logs. Customers are billed for the compute resources consumed in the form of Direct Query OpenSearch Compute Units (OCUs; usage rates will apply). After configuring the new data source, customers can start querying their data directly from the OpenSearch API or OpenSearch Dashboards."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-does-amazon-opensearch-service-zero-etl-integration-with-amazon-s3-pricing-work-7","itemHeading":"How does Amazon OpenSearch Service zero-ETL integration with Amazon S3 pricing work?","itemLongLoc":"
Customers only pay for the resources consumed by their workload. OpenSearch Service charges for only the compute needed to directly query external data as well as maintain optional indexes in OpenSearch Service. The compute capacity is measured in OpenSearch Compute Units (OCUs) which is the same units used by Amazon OpenSearch Serverless and Amazon OpenSearch Ingestion. The number of OCUs corresponds directly to the vCPU and memory required to query or maintain indexes based on the data. Customers will see one entry for compute in OCU-hours with the label for direct query. OCUs are billed on an hourly basis with per-minute granularity. If no queries or indexing activities are active, no OCUs are consumed. Costs for Amazon S3 or AWS Glue Data Catalog will be billed separately in the customer’s account. For more pricing details visit the Amazon OpenSearch Service pricing page. "},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#what-integration-options-does-amazon-opensearch-service-have-with-amazon-security-lake-8","itemHeading":"What integration options does Amazon OpenSearch Service have with Amazon Security Lake?","itemLongLoc":"
The integration between Amazon OpenSearch Service and Amazon Security Lake provides a streamlined experience for directly searching, gaining insights from, and analyzing data stored in Security Lake, all within the OpenSearch Service. There are two ways you can integrate Security Lake and OpenSearch Service: query-access and data-access. \n
With the query-access integration, security analysts can use the OpenSearch Dashboards console to run direct queries on the data in Security Lake and leverage on-demand indexing and pre-built content to get started with security-related queries. For the data-access integration, the OpenSearch ingestion service provides a preconfigured Security Lake blueprint, which includes a default configuration for ingesting Open Cybersecurity Schema Framework (OCSF) parquet files from Security Lake. The zero-ETL integration with Security Lake is only offered on collections with OpenSearch UI as opposed to on clusters."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topics#how-do-i-get-started-with-amazon-opensearch-service-and-security-lake-integration-9","itemHeading":"How do I get started with Amazon OpenSearch Service and Security Lake integration?","itemLongLoc":"
To get started, you first need to have an existing Security Lake setup in your AWS environment. This will provide the centralized storage and access to your enterprise security data. \n
You can then enable the integration between the two services through the AWS Management Console or by using AWS CloudFormation templates. This involves configuring the necessary permissions and access controls to allow Amazon OpenSearch Service to securely access and query the data in your Security Lake. \n
You can then explore the pre-built queries and integrations available through OCSF to quickly get started with common security analytics use cases. You also have the option to configure on-demand indexing of specific data sets from your Security Lake into OpenSearch Service for advanced analytics and visualization needs."},"metadata":{"tags":[{"name":"Zero-ETL Integrations","namespaceId":"awt-content-topics#ams#c121","id":"awt-content-topics#ams#c121#zero-etl-integrations-10"}]}},{"fields":{"itemDark":"false","id":"awt-content-topicsc1","itemHeading":"NA","itemLongLoc":"
NA","patternLongLoc":"
\n
General \n
Setup and configuration \n
Administration \n
Multi-AZ with Standby \n
Service Level Agreement \n
Cross-cluster search \n
Cross-cluster replication \n
Name change \n
Upgrades \n
Extended Support \n
Zero-ETL Integrations \n"},"metadata":{"tags":[]}}]},"metadata":{"auth":{},"testAttributes":{}},"context":{"page":{},"contentType":null,"environment":{"stage":"prod","region":"us-west-2"},"sdkVersion":"2.0.9"},"refMap":{"manifest.js":"08d0b0adb7","rt-groupable-faq.rtl.css":"1590d9a5ba","rt-groupable-faq.js":"8acc355ca5","rt-groupable-faq.css.js":"b89cb952e0","rt-groupable-faq.css":"f2695c009c","rt-groupable-faq.rtl.css.js":"96251c28ce"},"settings":{"templateMappings":{"dark":"itemDark","topic":"itemHeading","content":"itemLongLoc","grouping":"patternLongLoc"}}}

Page topics

General
9

Setup and configuration
19

Administration
47

Multi-AZ with Standby
6

Service Level Agreement
2

Cross-cluster search
7

Cross-cluster replication
5

Name change
4

Upgrades
11

Extended Support
10

Zero-ETL Integrations
10

General
Open all

What is Amazon OpenSearch Service?

Amazon OpenSearch Service is a managed service that makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. OpenSearch is an open source, distributed search and analytics suite derived from Elasticsearch. Amazon OpenSearch Service offers the latest versions of OpenSearch, support for 19 versions of Elasticsearch (1.5 to 7.10 versions), as well as visualization capabilities powered by OpenSearch Dashboards and Kibana (1.5 to 7.10 versions). Amazon OpenSearch Service currently has tens of thousands of active customers with hundreds of thousands of clusters under management processing trillions of requests per month. See the Amazon OpenSearch Service FAQ for more information.

Which OpenSearch and Elasticsearch versions does Amazon OpenSearch Service support?

Amazon OpenSearch Service offers the latest versions of OpenSearch and support for several legacy open-source Elasticsearch versions (up to version 7.10). For more details refer our documentation.

What is an Amazon OpenSearch Service domain?

Amazon OpenSearch Service domains are Elasticsearch (1.5 to 7.10) or OpenSearch clusters created using the Amazon OpenSearch Service console, CLI, or API. Each domain is an OpenSearch or Elasticsearch cluster in the cloud with the compute and storage resources you specify. You can create and delete domains, define infrastructure attributes, and control access and security. You can run one or more Amazon OpenSearch Service domains.

What does Amazon OpenSearch Service manage on my behalf?

Amazon OpenSearch Service manages the work involved in setting up a domain, from provisioning infrastructure capacity in the network environment you request to installing the OpenSearch or Elasticsearch software. Once your domain is running, Amazon OpenSearch Service automates common administrative tasks, such as performing backups, monitoring instances and patching software. Amazon OpenSearch Service integrates with Amazon CloudWatch to produce metrics that provide information about the state of the domains. Amazon OpenSearch Service also offers options to modify your domain instance and storage settings to simplify the task of tailoring your domain based to your application needs.

Does Amazon OpenSearch Service support the open-source Elasticsearch and OpenSearch APIs?

Amazon OpenSearch Service supports most of the commonly used OpenSearch and Elasticsearch APIs, so the code, applications, and popular tools that you're already using with your Elasticsearch (up to version 7.10) or OpenSearch environments work seamlessly. For a full list of supported operations, see our documentation.

What are the Availability Zone (AZ) deployment options available on Amazon OpenSearch Service?

Amazon OpenSearch Service offers customers the option to deploy their instances across one, two, or three AZs. Customers running development or test workloads can pick the single AZ option. Those running production-grade workloads should use two or three AZs. Three AZ deployments are strongly recommended for workloads with higher availability requirements.

Note: The three AZ option is only available in regions where there are three or more AZs.

In which regions does Amazon OpenSearch Service offer three AZ deployments?

Amazon OpenSearch Service supports three AZ deployments in all regions in which the service is available, except US West (N. California), where we support two AZs only.

How can Amazon OpenSearch Service be used for resources running on premises or on other clouds?

Amazon OpenSearch Service is a fully managed service that lets you run and scale OpenSearch clusters without having to worry about managing, monitoring, and maintaining your infrastructure, or having to build in-depth expertise in operating OpenSearch clusters. As a fully managed service, Amazon OpenSearch Service currently runs on AWS. However, OpenSearch is a distributed, community-driven, Apache 2.0-licensed, 100% open-source search and analytics suite that you can run on premises or in hybrid and multicloud environments. For example, there are partners who provide OpenSearch on other cloud platforms, or use OpenSearch in their applications. OpenSearch helps you more easily ingest, secure, search, aggregate, view, and analyze data for a number of use cases such as log analytics, application search, enterprise search, and more. OpenSearch provides a highly scalable system for providing fast access and response to large volumes of data with an integrated visualization tool—OpenSearch Dashboards—that makes it easier for users to explore their data. OpenSearch is powered by the Apache Lucene search library, and it supports a number of search and analytics capabilities such as k-nearest neighbors (KNN) search, SQL, Anomaly Detection, Machine Learning Commons, Trace Analytics, full-text search, and more.

Does size flexibility apply to Opensearch Reserved nodes?

No, Opensearch Reserved instance are not flexible; they only apply to the exact instance type that you reserve.

Setup and configuration
Open all

Can I create and modify my Amazon OpenSearch Service domain through the Amazon OpenSearch Service console?

Yes. You can create a new Amazon OpenSearch Service domain with the Domain Creation Wizard in the console with just a few clicks. While creating a new domain you can specify the number of instances, instance types, and EBS volumes you want allocated to your domain. You can also modify or delete existing Amazon OpenSearch Service domains using the console.

Does Amazon OpenSearch Service support Amazon VPC?

Yes, Amazon OpenSearch Service is integrated with Amazon VPC. When choosing VPC access, IP addresses from your VPC are attached to your Amazon OpenSearch Service domain and all network traffic stays within the AWS network and is not accessible to the Internet. Moreover, you can use security groups and IAM policies to restrict access to your Amazon OpenSearch Service domains.

Can I use CloudFormation Templates to provision Amazon OpenSearch Service domains?

Yes. AWS CloudFormation supports Amazon OpenSearch Service. For more information, see the CloudFormation Template Reference documentation.

Does Amazon OpenSearch Service support configuring dedicated master nodes?

Yes. You can configure dedicated master nodes for your domains. When choosing a dedicated master configuration, you can specify the instance type and instance count.

Can I create multiple Elasticsearch or OpenSearch indices within a single Amazon OpenSearch Service domain?

Yes. You can create multiple Elasticsearch or OpenSearch indices within the same Amazon OpenSearch Service domain. Elasticsearch and OpenSearch automatically distribute the indices and any associated replicas between the instances allocated to the domain.

How do I ingest data into my Amazon OpenSearch Service domain?

Amazon OpenSearch Service supports three options for data ingestion:

For large data volumes, we recommend Amazon Kinesis Data Firehose, a fully managed service that automatically scales to match the throughput of your data and requires no ongoing administration. It can also transform, batch and compress the data before loading it.

Amazon OpenSearch Service supports integration with Logstash. You can configure your Amazon OpenSearch Service domain as the data store for all logs arriving from your Logstash implementation.

You can use native Elasticsearch (up to version 7.10) or OpenSearch APIs, such as the index and bulk APIs, to load data into your domain.

Does Amazon OpenSearch Service support integration with Logstash?

Yes. Amazon OpenSearch Service supports integration with Logstash. You can set up your Amazon OpenSearch Service domain as the backend store for all logs coming through your Logstash implementation. You can set up access control on your Amazon OpenSearch Service domain to either use request signing to authenticate calls from your Logstash implementation, or use resource based IAM policies to include IP addresses of instances running your Logstash implementation.

Does Amazon OpenSearch Service support integration with Kibana?

Yes. Amazon OpenSearch Service offers visualization capabilities powered by OpenSearch Dashboards and Kibana (1.5 to 7.10 versions).

What storage options are available with Amazon OpenSearch Service?

You can choose between local on-instance storage or EBS volumes. During domain creation, if you select EBS storage, you can increase and decrease the size of the storage volume as necessary.

What types of EBS volumes does Amazon OpenSearch Service support?

You can choose between Magnetic, General Purpose, and Provisioned IOPS EBS volumes.

Is there a limit on the amount of storage that can be allocated to an Amazon OpenSearch Service domain?

Yes. Amazon OpenSearch Service deploys storage based the choice of instance and/or the size of the associated EBS volume. The maximum storage per node is 24 TB with R6g.12xlarge instances with EBS gp3 storage. With the default maximum of 80 data nodes allowed per Amazon OpenSearch Service domain, you can allocate about 1920 TB of storage to a single domain. You can request a service limit increase up to 200 instances per domain by creating a case with the AWS Support Center. With 200 instances, you can allocate about 3 PB of storage to a single domain.

How are dedicated master instances distributed across AZs?

If you deploy your data instances in a single AZ, your dedicated master instances are also deployed in the same AZ. However, if you deploy your data instances across two or three AZs, Amazon OpenSearch Service automatically distributes the dedicated master instances across three AZs. The exception to this rule occurs if a region only has two AZs or if you select an older-generation instance type for the master instances that is not available in all AZs. For more details, refer our documentation.

What is the recommended AZ configuration for production workloads?

For production workloads, we recommend deploying your data instances across three AZs since it offers better availability. Also, we recommend provisioning instances in multiples of three for equal distribution across AZs. In regions where three AZs are not available, we recommend using a two AZ deployment with an even number of data instances. In all cases, we recommend provisioning three dedicated master instances.

How can I configure my domain for three AZ deployment?

You can enable three AZ deployment for both existing and new domains using the AWS console, CLI or SDKs. For more details, refer our documentation.

Is there a fee for enabling three AZ deployment?

No. Amazon OpenSearch Service does not charge anything for enabling three AZ deployment. You only pay for the number of instances in your domain, not the number of AZs to which they are deployed.

I no longer see the “zone awareness” option in my console. Is my domain no longer zone aware?

All domains configured for multiple AZs will have zone awareness enabled to ensure shards are distributed across Availability Zones. In the console, you can now explicitly choose two or three AZ deployments. Domains previously configured with “Zone Awareness” will continue to be deployed across two AZs unless they are reconfigured. For more details, refer our documentation.

How does Amazon OpenSearch Service handle instance failures and AZ disruptions?

If one or more instances in an AZ are unreachable or not functional, Amazon OpenSearch Service automatically tries to bring up new instances in the same AZ to replace the affected instances. In the rare event that new instances cannot be brought up in the AZ, Amazon OpenSearch Service brings up new instances in the other available AZs if the domain has been configured to deploy instances across multiple AZs. Once the AZ issue resolves, Amazon OpenSearch Service rebalances the instances such that they are equally distributed across the AZs configured for the domain. For more details refer our documentation.

If I have only one replica for the indices in my domain, should I use two or three AZs?

Even when you configure one replica, we recommend three AZs. If an AZ disruption occurs in a three AZ domain, you only lose one-third of your capacity but if the disruption occurs in a two AZ domain, you lose half your capacity, which can be more disruptive. Also, in a three AZ domain, when an AZ is disrupted, Amazon OpenSearch Service can fall back to the two remaining AZs, and still support cross-AZ replication . In a two AZ domain, you lose cross-AZ replication if one AZ is disrupted, which can further reduce availability. For more details refer our documentation.

How do I leverage three AZ deployment for my VPC domain?

The number of AZs your domain is deployed to corresponds to the number of subnets you have configured for your VPC domain. You need to configure at least three subnets in your VPC domain to enable three AZ deployment. For more details on configuring VPC, refer our documentation.

Administration
Open all

Can programs running on servers in my own data center access my Amazon OpenSearch Service domains?

Yes. The programs with public Internet access can access Amazon OpenSearch Service domains through a public endpoint. If your data center is already connected to Amazon VPC through Direct Connect or SSH tunneling, you can also use VPC access. In both cases, you can configure IAM policies and security groups to allow programs running on servers outside of AWS to access your Amazon OpenSearch Service domains. Click here for more information about signed requests.

How can I migrate data from my existing OpenSearch/Elasticsearch cluster to a new Amazon OpenSearch Service domain?

To migrate data from an existing Elasticsearch or OpenSearch cluster, you should create a snapshot of an existing cluster, and store the snapshot in your Amazon S3 bucket. Then you can create a new Amazon OpenSearch Service domain and load data from the snapshot into the newly created Amazon OpenSearch Service domain using the restore API.

How can I scale an Amazon OpenSearch Service domain?

Amazon OpenSearch Service allows you to control the scaling of your Amazon OpenSearch Service domains using the console, API, and CLI. You can scale your Amazon OpenSearch Service domain by adding, removing, or modifying instances or storage volumes depending on your application needs. Amazon OpenSearch Service is integrated with Amazon CloudWatch to provide metrics about the state of your Amazon OpenSearch Service domains to enable you to make appropriate scaling decisions for your domains.

Does scaling my Amazon OpenSearch Service domain require downtime?

No. Scaling your Amazon OpenSearch Service domain by adding or modifying instances, and storage volumes is an online operation that does not require any downtime.

Does Amazon OpenSearch Service support cross-zone replication?

Yes. If you enable replicas for your OpenSearch/Elasticsearch indices and use multiple Availability Zones, Amazon OpenSearch Service automatically distributes your primary and replica shards across instances in different AZs.

Does Amazon OpenSearch Service expose any performance metrics through Amazon CloudWatch?

Yes. Amazon OpenSearch Service exposes several performance metrics through Amazon CloudWatch including number of nodes, cluster health, searchable documents, EBS metrics (if applicable), CPU, memory and disk utilization for data and master nodes. Please refer to the service documentation for a full listing of available CloudWatch metrics.

I wish to perform security analysis or operational troubleshooting of my Amazon OpenSearch Service deployment. Can I get a history of all the Amazon OpenSearch Service API calls made on my account?

Yes. AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The AWS API call history produced by AWS CloudTrail enables security analysis, resource change tracking, and compliance auditing. Learn more about AWS CloudTrail at the AWS CloudTrail detail page, and turn it on via CloudTrail's AWS Management Console home page.

What is a snapshot?

A snapshot is a copy of your Amazon OpenSearch Service domain at a moment in time.

Why would I need snapshots?

Creating snapshots can be useful in case of data loss caused by node failure, as well as the unlikely event of a hardware failure. You can use snapshots to recover your Amazon OpenSearch Service domain with preloaded data or to create a new Amazon OpenSearch Service domain with preloaded data. Another common reason to use backups is for archiving purposes. Snapshots are stored in Amazon S3.

Does Amazon OpenSearch Service provide automated snapshots?

Yes. By default, Amazon OpenSearch Service automatically creates hourly snapshots of each Amazon OpenSearch Service domain and retains them for 14 days.

How long are the automated daily hourly snapshots stored by Amazon OpenSearch Service?

Amazon OpenSearch Service will retain the last 14 days worth of automated hourly snapshots.

Is there a charge for the automated daily hourly snapshots?

There is no additional charge for the automated hourly snapshots. The snapshots are stored for free in an Amazon OpenSearch Service S3 bucket and will be made available for node recovery purposes.

Can I create additional snapshots of my Amazon OpenSearch Service domains as needed?

Yes. You can use the snapshot API to create additional manual snapshots in addition to the daily-automated snapshots created by Amazon OpenSearch Service. The manual snapshots are stored in your S3 bucket and will incur relevant Amazon S3 usage charges.

Can snapshots created by the manual snapshot process be used to recover a domain in the event of a failure?

Yes. Customers can create a new Amazon OpenSearch Service domain and load data from the snapshot into the newly created Amazon OpenSearch Service domain using the OpenSearch/Elasticsearch restore API.

What happens to my snapshots when I delete my Amazon OpenSearch Service domain?

The daily snapshots retained by Amazon OpenSearch Service will be deleted as part of domain deletion. Before deleting a domain, you should consider creating a snapshot of the domain in your own S3 buckets using the manual snapshot process. The snapshots stored in your S3 bucket will not be affected if you delete your Amazon OpenSearch Service domain.

What types of OpenSearch/Elasticsearch logs are exposed by Amazon OpenSearch Service?

Amazon OpenSearch Service exposes three Elasticsearch or OpenSearch logs through Amazon CloudWatch Logs: error logs, search slow logs, and index slow logs. These logs are useful for troubleshooting performance and stability issues with one’s domain.

What are slow logs?

Slow logs are log files that help track the performance of various stages in an operation. OpenSearch and Elasticsearch exposes two kinds of slow logs:

Index Slow Logs – These logs provide insights into the indexing process and can be used to fine-tune the index setup.

Search Slow Logs – These logs provide insights into how fast or slow queries and fetches are performing. These logs help fine tune the performance of any kind of search operation on OpenSearch or Elasticsearch.

For complete details on slow logs, please refer to OpenSearch documentation.

How can I enable slow logs on Amazon OpenSearch Service?

Slows logs can be enabled via the click of a button from the Console or via our CLI and APIs. For more details please refer to our documentation .

Can I only enable slow logs for specific indices?

Yes. You can update the settings for a specific index to enable or disable slow logs for it. For more details refer to our documentation.

Does turning on slow logs in Amazon OpenSearch Service automatically enable logging for all indexes?

No. Turning on slow logs in Amazon OpenSearch Service enables the option to publish the generated logs to Amazon CloudWatch Logs for indices in the given domain. However, in order to generate the logs you have to update the settings for one or more indices to start the logging process. For more details on setting the index configuration for enabling slow logs, please refer to our documentation.

If I turn off the slow logs in Amazon OpenSearch Service, does it mean that log files are no longer being generated?

No. The generation of log files are dependent on the index settings. To turn off generation of the log files you have to update the index configuration. For more details on setting the index configuration for enabling slow logs, see our documentation.

Can I change the granularity of logging?

You can only change the granularity of logging for slow logs. OpenSearch and Elasticsearch expose multiple levels of logging for slow logs. You need to set the appropriate level in the configuration of your index. For more details on setting the index configuration for enabling slow logs, please refer to OpenSearch documentation.

Will enabling slow logs or error logs cost me anything?

When slow logs or error logs are enabled, Amazon OpenSearch Service starts publishing the generated logs to CloudWatch Logs. Amazon OpenSearch Service does not charge anything for enabling the logs. However, standard CloudWatch charges will apply.

What kinds of error logs are exposed by Amazon OpenSearch Service?

OpenSearch uses Apache Log4j 2 and its built-in log levels (from least to most severe) of TRACE, DEBUG, INFO, WARN, ERROR, and FATAL. If you enable error logs, Amazon OpenSearch Service publishes log lines of WARN, ERROR, and FATAL, and select errors from the DEBUG level to CloudWatch. For more details, refer our documentation.

How can I enable error logs on Amazon OpenSearch Service?

Error logs can be enabled via the click of a button from the AWS Console or via our CLI and APIs. For more details please refer to our documentation.

Can I enable error logs for only specific indices?

No, error logs are exposed for the entire domain. That is, once enabled, log entries from all indices in the domain will be made available.

Are error logs available for all versions of Elasticsearch supported by Amazon OpenSearch Service?

No, error logs are available only for Elasticsearch versions 5.x and above.

Is there any limit on the size of each log entry?

Yes. Each log entry made into CloudWatch will be limited to 255,000 characters. If your log entry is bigger than that, it will be truncated to 255,000 characters.

What is the recommended best practice for using slow logs?

Slow logs are only needed when you want to troubleshoot your indexes or fine-tune performance. The recommended approach is to only enable logging for those indexes for which you need additional performance insights. Also, once the investigation is done, you should turn off logging so that you don’t incur any additional costs on account of it. For more details, see our documentation.

How can I consume logs from CloudWatch Logs?

CloudWatch offers multiple ways to consume logs. You can view log data, export it to S3, or process it in real time. To learn more, see the CloudWatch Logs developer guide.

Are slow logs available for all versions of OpenSearch and Elasticsearch supported by Amazon OpenSearch Service?

Yes, slow logs can be enabled for all versions of OpenSearch and Elasticsearch supported by Amazon OpenSearch Service. However, there are slight differences in the way log settings can be specified for each version of Elasticsearch. Please refer to our documentation for more details.

Will the cluster have any down time when logging is turned on or off?

No. There will not be any down-time. Every time the log status is updated, we will deploy a new cluster in the background and replace the existing cluster with the new one. This process will not cause any down time. However, since a new cluster is deployed the update to the log status will not be instantaneous.

Which Elasticsearch and OpenSearch versions does the in-place upgrade feature support?

Amazon OpenSearch Service currently supports in-place version upgrade for domains with any OpenSearch version or Elasticsearch versions 5.x and above. The target versions that we support for the upgrade are 5.6, 6.3, 6.4, 6.5, 6.7, 6.8, 7.1, 7.4, 7.7, 7.8, 7.9, and 7.10. For more details refer our documentation.

My domain runs a version of Elasticsearch older than 5.x. How do I upgrade those domains?

Please refer to our documentation for details on migrating from various Elasticsearch versions.

Will my domain be offline while the in-place upgrade is in progress?

No. Your domain remains available throughout the upgrade process. However, part of the upgrade process involves relocating shards, which can impact domain performance. We recommend upgrading when the load on your domain is low.

How can I check if my domain’s Elasticsearch version can be upgraded?

In-place version upgrade is available only for domains running Elasticsearch 5.x and above. If your domain is of version 5.x or above, you can run the upgrade eligibility check to validate whether your domain can be upgraded to the desired version. Please refer to our documentation to learn more.

What are the tests done by Amazon OpenSearch Service to validate my domains upgrade eligibility?

For detailed list of the tests we run to validate upgrade eligibility, please refer to our documentation.

Can I update my domain configuration while the version upgrade is in progress?

No. Once the in-place version upgrade has been triggered, you cannot make changes to your domain configuration until the upgrade completes or fails. You can continue reading and writing data while the upgrade is in progress. Also, you can delete the domain, in which case the upgrade is terminated and the domain deleted.

What happens to the automated system snapshot when the in-place version upgrade is in progress?

The version upgrade process automatically takes a snapshot of the system and only starts the actual upgrade if the snapshot succeeds. If the upgrade is in progress when the automated snapshot’s start time is reached, the automated snapshot is skipped for that day and continued on the next day.

How does Amazon OpenSearch Service safeguard against issues that can crop up during version upgrades?

Amazon OpenSearch Service runs a set of tests before triggering the upgrade to check for known issues that can block the upgrade. If no issues are encountered, the service takes a snapshot of the domain and starts the upgrade process if the snapshot is successful. The upgrade is not triggered if there are any issues encountered with any of the steps.

What happens if the system encounters issues while performing the in-place version upgrade?

If encountered issues are minor and fixable, Amazon OpenSearch Service automatically tries to address them and unblock the upgrade. However, if an issue blocks the upgrade, the service reverts back to the snapshot that was taken before the upgrade and logs the error. For more details on viewing the logs from the upgrade progress, please refer to our documentation.

Can I view the history of upgrades on my domain?

Yes. You can view the upgrade logs from the AWS console or request them using the CLI or SDKs. Please refer to our documentation for more details.

Can I pause or cancel the version upgrade after it has been triggered?

No. After the upgrade has been triggered, it cannot be paused or cancelled until it either completes or fails.

Can I run in-place version upgrade on multiple domains in parallel?

Yes. However, if you want to keep all of your domains on the same version, we recommend running the upgrade eligibility check on all domains before upgrading them. This extra step can help catch issues with one domain that might not be present on others.

How long does the in-place version upgrade take?

Depending on the amount of data and the size of the cluster, upgrades can take anywhere from a few minutes to a few hours to complete.

Can I just upgrade the domain quickly without retaining any of the data?

No. With in-place version upgrade, all the data in your cluster is also restored as part of the upgrade process. If you only wish to upgrade the domain alone, you can take a snapshot of your data, delete all your indexes from the domain and then trigger an in-place version upgrade. Alternatively, you can create a separate domain with the newer version and then restore your data to that domain.

Can I downgrade to previous version if I’m not comfortable with the new version?

No. If you need to downgrade to an older version, contact AWS Support to restore the automatic, pre-upgrade snapshot on a new domain. If you took a manual snapshot of the original domain, you can perform this step yourself.

Multi-AZ with Standby
Open all

What is Multi-AZ with Standby?

Multi-AZ with Standby is a new deployment option for Amazon OpenSearch Service that enables high-availability and consistent performance for business-critical workloads. With Multi-AZ with Standby, OpenSearch Service managed clusters are resilient to infrastructure failures like node drops, or a single Availability Zone failure, assuring no impact to performance or availability, even in the event of a single Availability Zone failure. Multi-AZ with Standby provides the added benefit of simplifying cluster configuration and management by enforcing best practices and reducing complexity.

What are the pre-requisites for creating or updating a cluster with Multi-AZ with Standby?

To enable Multi-AZ with Standby, managed clusters need to meet the following conditions:

Run OpenSearch 1.3 or more recent version.

Deploy in AWS Regions with 3-AZ. Currently, AWS North California region does not support 3-AZ and are therefore not suitable for Multi-AZ with Standby.

Number of data nodes needs to be in multiples of three.

Number of data copies (primary + replica) should be in multiples of three.

Follow sizing guidelines for the leader (recommended size based on number of nodes, number of shards, and number of mappings in your cluster).

What are the failure scenarios covered and not covered by the feature?

With Multi-AZ with Standby, Amazon OpenSearch Service detects and automatically recovers from some of the infrastructure failures. Amazon OpenSearch Service automatically fails over from active to standby nodes in under a minute when any of the following events occur:

Loss of one active AZ or all nodes in an active AZ

Loss of connectivity to one active AZ

Instance hardware failure in the active AZ

Storage failure on a node in the active AZ

Currently Multi-AZ with Standby does not cover the following events:

Loss of master Quorum, as recovery from this event can take several minutes

Loss of multiple Availability Zones

Loss of connectivity to a Region

Loss of more than 50% nodes in more than one AZ

Downtime caused due to insufficient compute or storage as a result of change in workload characteristics

Downtime caused due to rouge queries

Loss of one or more services that Amazon OpenSearch Service depends on like ARPS and ALB

Downtime of OpenSearch Dashboard during version upgrades

Do managed clusters that use Multi-AZ with Standby need to be sized differently? How do we size managed clusters for Multi-AZ with Standby?

No. In principle, the sizing guidelines remain the same. Multi-AZ with Standby has pre-requisites that simplify the mental model required to size the cluster. The way to think of sizing for managed cluster is that you should look at the capacity needed to service your workload and then add 50% for redundancy. The primary difference between the current ‘Zone Awareness’ option and the Multi-AZ with Standby option is how redundant or additional capacity is handled to maintain availability. Multi-AZ with Standby requires you have at least one copy of data in each AZ, so that Multi-AZ with Standby can explicitly reserve capacity in one AZ as standby. This standby capacity acts as a failover target during AZ disruption or instance failure. The existing model requires that you maintain an optimum levels of resources to serve your workload. You would continue to monitor your cluster for sizing issues and take corrective actions as the workload characteristics changes.

Does choosing Multi-AZ with Standby mean I no longer have to ensure my cluster is properly sized and resourced for my workload?

No. Amazon OpenSearch Service works on a shared responsibility model. You are responsible for ensuring that your cluster is sized in accordance with your workload. Multi-AZ with Standby make the mental model of setting up your cluster simple. You should continue to monitor the error and latency metrics along with storage, CPU and RAM utilization for signals that the cluster is overloaded and may need to be scaled.

Will I incur additional cost if I use Multi-AZ with Standby?

No. The feature Multi-AZ with Standby is available at no extra cost. You continue to pay for the resources deployed on the cluster to serve your workload. If your cluster already follows the best practices and has at least three copies of data for a 3-AZ cluster, you are unlikely to incur additional cost in moving to Multi-AZ with Standby. However, if your cluster is undersized or does not have enough redundant capacity to serve your workload, you will need to add capacity to move to Multi-AZ with Standby for enhanced availability and performance. The standby capacity is reserved from the configured total capacity.

Service Level Agreement
Open all

What does the Amazon OpenSearch Service SLA guarantee?

Our Amazon OpenSearch Service SLA guarantees a Monthly Uptime Percentage of at least 99.9% for Amazon OpenSearch Service.

How do I know if I qualify for a SLA Service Credit?

You are eligible for a SLA credit for Amazon OpenSearch Service under the Amazon OpenSearch Service SLA if multi-AZ domains on Amazon OpenSearch Service have a Monthly Uptime Percentage of less than 99.9% during any monthly billing cycle.

For full details on all of the terms and conditions of the SLA, as well as details on how to submit a claim, please see the Amazon OpenSearch Service SLA details page.

Cross-cluster search
Open all

What is cross-cluster search?

Cross-cluster search is an Elasticsearch and OpenSearch feature that enables performing queries and aggregation across two connected clusters. It works by setting up a lightweight unidirectional connection between participating clusters.

What are the minimum requirements for a domain to participate in cross-cluster search?

Domains participating in a cross-cluster search needs to meet the following criteria:

Participating domains should be on OpenSearch or Elasticsearch version 6.8 and above

Participating domains need to have encryption in transit enabled

Participating domains need to have Fine Grained Access Control (FGAC) enabled

Participating domains versions should adhere to the same rules as rolling version upgrade

What are the instance types that support cross-cluster search?

Cross-cluster search is currently supported on the following instance types

i2, i3 family

r3, r4, r5 family

m4, m5 family

c4, c5, family

Graviton family

What are the instance types that do not support cross-cluster search?

Cross-cluster search is not supported on the t2 and m3 family instances due to technical limitation.

Can domains in two different AWS accounts participate in cross-cluster search?

Yes. Participating domains can belong to two different AWS accounts.

Can domains in two different AWS regions participate in cross-cluster search?

Yes.

How can I start using cross-cluster search?

To get started with cross-cluster search, follow the documentation here.

Cross-cluster replication
Open all

What is cross-cluster replication?

Cross-cluster replication, a new capability that allows Amazon OpenSearch Service customers to automate copying and synchronizing indices from one cluster to another at low latency in same or different AWS Regions.

What are the minimum requirements for a domain to participate in cross-cluster replication?

Domains participating in a cross-cluster replications needs to meet the following criteria:

Participating domains should be on Elasticsearch version 7.10

Participating domains need to have encryption in transit enabled

Participating domains need to have Fine Grained Access Control (FGAC) enabled

Participating domains versions should adhere to the same rules as rolling version upgrade

Can domains in two different AWS Regions participate in cross-cluster replication?

Yes. domains in two different AWS Regions can participate in cross-cluster replication.

Does cross-cluster replication support Ultrawarm and Cold Storage?

No. Current implementation of cross-cluster replication does not support Ultrawarm or Cold Storage.

What are the charges for cross-cluster replication?

Yes. You need to pay standard AWS data transfer charges for the data transferred in and out of Amazon OpenSearch Service.

Name change
Open all

Why did the name change to Amazon OpenSearch Service from Amazon Elasticsearch Service?

We announced the OpenSearch project, a community-driven, open source fork of Elasticsearch and Kibana, on April 12, 2021. We committed to making a long-term investment in OpenSearch to ensure users continue to have a secure, high-quality, fully open source search and analytics suite with a rich roadmap of new and innovative functionality. This project includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2). We launched version 1.0 of OpenSearch on July 12, 2021. As part of our long-term commitment to OpenSearch, we added support for OpenSearch 1.0 on the managed service on September 7, 2021 and changed the name from Amazon Elasticsearch Service to Amazon OpenSearch Service. Along with OpenSearch 1.0, we continue to support legacy Elasticsearch versions until 7.10 on the service. Aside from the name change, you can rest assured that we will continue to deliver the same great experience without any impact to ongoing operations, development methodology, or business use. Learn more about OpenSearch here: https://opensearch.org

Do I, as a customer, have to take any action as part of this name change?

We have strived to make this name change as seamless as possible for you. There are aspects, such as the new SDK/configuration APIs, that require your action to ensure you derive the best benefits from the service. While the existing SDK will continue to work from a compatibility perspective, any new functionality that requires new configuration APIs will only be implemented in the new SDK. Hence, we recommend that you move to the new SDK. In addition, irrespective of the new SDK, we strongly recommend that you move your existing IAM policies to use the renamed configuration APIs. As of now, your existing IAM policies will continue to work with the old API definition. However, we will move over to the new API-based permission validation and we will eventually require you to use the new APIs in your policies (specifically for the APIs where there is a name change; e.g. CreateElasticsearchDomain to CreateDomain). Please see the documentation for more details.

Do I have to move to the new SDK to upgrade to OpenSearch 1.0?

No. From a backward compatibility perspective, we will ensure that your existing setup continues to work with OpenSearch 1.0. However, we recommend that you eventually move to the latest SDK for a cleaner and up-to-date experience, as mentioned above.

Are there any changes to pricing with this name change?

No. There are no changes to pricing.

OpenSearch includes certain Apache-licensed Elasticsearch code from Elasticsearch B.V. and other source code. Elasticsearch B.V. is not the source of that other source code. ELASTICSEARCH is a registered trademark of Elasticsearch B.V.

Upgrades
Open all

I'm using the Elasticsearch engine in Amazon OpenSearch Service. Why should I upgrade to the OpenSearch 1.x engine? What are the benefits for me?

Upgrading to OpenSearch 1.x ensures your search infrastructure is built on a growing and dynamic Apache-Licensed open source project and gives you access to a wealth of innovative improvements and features available in OpenSearch 1.2 (as of this writing). Features such as enterprise-grade security, alerting, data-lifecycle management, observability, ML-based anomaly detection, and more are all part of OpenSearch Service, with no additional licensing fees.

Will I have downtime if I upgrade?

We use a blue/green (BG) deployment process during upgrade. During a BG, the service adds nodes to the OpenSearch Service cluster in the new configuration and version, migrates data from the old nodes, and drops the old nodes when the data migration is complete. During a BG, search and indexing APIs are available and function normally. Although BG is designed not to interfere with query and indexing requests, some changes (especially those involving changes to security-related settings) can cause dashboards to be unavailable during the period of change.

What versions of OpenSearch and Elasticsearch does OpenSearch Service support?

OpenSearch supports several versions of Elasticsearch and OpenSearch, some of which already have end of Standard and Extended Support dates announced. For a complete list of engine versions, and corresponding support dates, please see the documentation. For more information about Extended Support, please see the Extended Support.

Is AWS deprecating older versions of Elasticsearch Service?

AWS maintains 19 versions of Apache-2.0-licensed Elasticsearch. None of these versions are deprecated or planned for deprecation at this time.

Will the upgrade trigger a BG? If not, what is the process for upgrading our nodes?

Yes, the upgrade will trigger a BG deployment process. Please review the upgrade preparation and steps here.

I want to move to Amazon OpenSearch Service 1.x to take advantage of AWS Graviton2 instances, but I am locked in with my existing reserved instances (RIs). How can you help?

Please work with your AWS account team for information based on your specific situation with RIs.

What should I plan for before initiating an upgrade to Amazon OpenSearch Service 1.x or greater?

The OpenSearch project 1.0 is a fork of open source Elasticsearch 7.10.2. It is wire-compatible with Elasticsearch 7.10--you don’t need to change your usage. To migrate, you can upgrade your domain to version Elasticsearch 7.10 from any prior version in the 6.x and 7.x series, take a snapshot, and restore that snapshot to a domain running OpenSearch Service 1.x. Some clients or tools include version checks that may cause the client or tool to not work with OpenSearch Service. When you upgrade, enable compatibility mode to work around these version checks.

Can I continue to use my existing clients, data collection, and data ingestion tools with Amazon OpenSearch Service 1.x?

In most cases, you can continue to use your existing clients. The APIs and core search functionality are compatible with Elasticsearch version 7.10.2. If you have older clients, the clients perform version checking, or the clients leverage functionality that is targeted to older versions of Elasticsearch such as major versions 5 or 6, then we suggest that you try to bring your clients up to a minimum standard of support on 7.10.2 to ensure a smooth transition.

The OpenSearch project supports a broad range of clients that are specifically built to address the OpenSearch versions of the engine on Amazon OpenSearch Service. Check your client against the list of the latest OpenSearch clients and supported programming languages for those clients.

You can enable the compatibility mode feature to interoperate with clients from other vendors, but remember to check the version that OpenSearch reports. Enable this setting to ensure the service responds with version 7.10.2 to clients developed prior to the introduction of the OpenSearch Service engine.

I'm running Elasticsearch version 5.x or earlier. What’s my best upgrade path?

Elasticsearch 5.x indices are not compatible with Elasticsearch 7.10 or OpenSearch 1.x. You must create a new index and load data from your source. If you are running a log analytics workload, you can evaluate whether your data retention strategy supports running in parallel while you build up a full data set on the new domain.

Are there any partners that can help me with my upgrade?

Yes, please contact [email protected] to request a list of partners for your region, industry, and project complexity. AWS Partner Network (APN) partners are trained and have the experience to assist you with upgrades.

Will Amazon OpenSearch Service remain compatible with Elasticsearch in the future? What’s the plan for the future?

OpenSearch 1.0 is a fork of Elasticsearch 7.10.2. OpenSearch and Elasticsearch are compatible. If you enable compatibility mode, Elasticsearch clients are also compatible with OpenSearch 1.0.

Amazon OpenSearch Service does not and will not offer versions of the Elasticsearch engine after version 7.10.2.

As AWS announced when we forked Elasticsearch, we intended to build, and have built a thriving community around OpenSearch. We have published a roadmap for OpenSearch, complete with community input and consensus on the feature priorities. We will make every reasonable effort to maintain compatibility with Elasticsearch. Our goal is to grow with our community and Amazon OpenSearch Service customers.

You can directly upgrade to OpenSearch Service 1.0 from Elasticsearch and Kibana versions 6.8.0 to 7.10.2, and open distro for Elasticsearch (ODFE) 1.x. For rolling upgrades from ODFE to OpenSearch, we recommend first upgrading to ODFE 1.13, then upgrading to OpenSearch 1.0.

Migration resources are here:

Analytics Migrations

Migrating to Amazon OpenSearch Service

Extended Support
Open all

What is Standard Support and Extended Support?

Every engine version that is launched in OpenSearch Service is by default covered by Standard Support. As part of Standard Support, AWS provides regular bug fixes and security updates. When Standard Support ends for a version, AWS provides an Extended Support period of at least 12 months after the Standard Support end date. During the Extended Support period, AWS will provide critical security fixes and operating system patches. This gives you more time to plan your upgrade to a more recent supported engine version. When you are running a version that is in Extended Support, you will be charged a flat fee/Normalized instance hour (NIH), in addition to the standard instance and storage costs. Please see the documentation for more information on Extended Support, and the schedule for various versions. For pricing information, please see the pricing page.

Do we need to opt-in for Extended Support?

No. Domains that are running versions that have reached the end of Standard Support will automatically be covered under Extended Support, and will be priced accordingly. Once you upgrade your domain to a newer version under Standard Support, you will stop being billed for Extended Support.

What are the charges for Extended Support and when do they start?

Domains that are running Extended Support will be charged an additional flat fee/Normalized instance hour (NIH), in addition to the standard instance and storage pricing. Please see pricing page for the exact pricing by region. Your domain will be charged for Extended Support automatically from the next day after end of Standard Support. If your domain is running a version that has Standard and Extended Support dates published (see here for details), we will send you a notification on your Personal Health Dashboard, on the OpenSearch Service console, and through EventBridge events, 3-months prior to the end of Standard Support date. For more information on monitoring notifications in OpenSearch Service, please see the documentation here.

How do I calculate Extended Support charges for my domains?

Domains running versions under Extended Support will be charged an additional flat fee/Normalized Instance Hour (NIH). NIH is computed as a factor of the instance size (e.g. medium, large), and the number of instance hours. For example, if you are running an m7g.medium.search instance for 24 hours in the US East (North Virginia), which is priced at $0.068/hour (on-demand), you will typically pay $1.632 ($0.068x24) as the instance cost. If you are running a version that is in Extended Support, you will pay an additional $0.0065/NIH, which is computed as $0.0065 x 24 (number of instance hours) x 2 (size normalization factor; 2 for medium-sized instances), which comes to $0.312 for Extended Support for 24 hours. The total amount you will pay for 24 hours will be a sum of the standard instance usage cost (excluding storage) and the Extended Support cost, which is $1.944 ($1.632+$0.312). Please see documentation for more information.

How do I avoid Extended Support charges?

You can upgrade your domain to an engine version that is covered by Standard Support. A version is covered under Standard Support until the published end of Standard Support date, or if no end of Standard Support dates have been announced for the version.

Can we extend support further beyond the Extended Support end date?

No. We recommend that you upgrade to a version that is covered by standard or Extended Support or to a version for which end of support has not been announced. Once Extended Support ends for a version, domains running the specific version will not receive bug fixes or security updates.

What happens after the end of Extended Support period?

Once Extended Support ends for a version, we will no longer release bug fixes or security updates for that specific version. Domains running unsupported versions will be isolated and data preserved for at least 30 days. Isolated domains can no longer be accessed, and you will have to restore data into a new domain running a supported version. We strongly recommend that you update your domain to a supported version before the end of Extended Support for a specific version. If you need further assistance, please contact AWS Support.

Are there any restrictions when a domain is running a version on Extended Support? For example, can I provision new instances or create new domains?

Yes. As long the engine version that you wish to use is covered by Extended Support, you can continue to operate the service as usual, without any restrictions.

What fixes are included in Extended Support?

As part of Extended Support, AWS provides critical security fixes, and operating system patches as required.

Do you support in-place upgrades to new versions without any downtime?

Yes, depending the version you are upgrading from and upgrading to. Please see the documentation here for a list of supported upgrade paths. When you are moving from legacy versions like ES 1.5 or ES 2.3, we do not support in-place upgrades. Please refer to the documentation here for instructions on upgrading your domains running legacy versions.

Zero-ETL Integrations
Open all

Why should I use the zero-ETL integration of Amazon OpenSearch Service with Amazon DynamoDB?

This zero-ETL integration with Amazon DynamoDB abstracts away the operational complexity in orchestrating the replication of data from an operational datastore to a search datastore. Data pipelines that are used to keep different datastores in sync can be challenging and costly to build and manage, and suffer from intermittent errors which are difficult to track. This integration enables Amazon DynamoDB customers to obtain near real-time search results from their transactional data by offering a fully managed solution for making operational data from Amazon DynamoDB available in Amazon OpenSearch Service within seconds of being written.

How does this zero-ETL integration replicate data from Amazon DynamoDB to Amazon OpenSearch Service?

The zero-ETL integration of Amazon OpenSearch Service with Amazon DynamoDB uses Amazon OpenSearch Ingestion to seamlessly move operational data from Amazon DynamoDB to Amazon OpenSearch Service. To enable an integration, customers first choose the Amazon DynamoDB table whose data needs to be replicated. The zero-ETL integration feature sets up an Amazon OpenSearch Ingestion pipeline in the customer’s account that takes care of replicating the data to an Amazon OpenSearch Service managed cluster or serverless collection. Amazon OpenSearch Ingestion understands the structure of the Amazon DynamoDB tables and then bootstraps an Amazon OpenSearch Service managed cluster or serverless collection with the existing data from the DynamoDB tables. Optionally, customers can specify a schema for the indices that will be created in Amazon OpenSearch Service. Any updates to the DynamoDB table are also replicated to Amazon OpenSearch Service without any manual intervention by customers.

How does data transformation work from while moving data from Amazon DynamoDB to Amazon OpenSearch Service?

This zero-ETL features uses Amazon OpenSearch Ingestion to move the data from Amazon DynamoDB to Amazon OpenSearch Service and leverages the native data transformational capabilities of Amazon OpenSearch Ingestion pipelines to aggregate and filter the data while it is in motion. When moving the data from an Amazon DynamoDB table, customers may want to drop a few fields or create new fields based on aggregations across existing fields. Optionally, customers can also write custom logic for Amazon OpenSearch Ingestion to achieve bespoke transformational capability. For other users, who just want to move their entire data from source to sink, Amazon OpenSearch Ingestion provides out-of-the box blueprints so that they can perform the integrations with just a few button clicks.

What security permissions are required for using the zero-ETL integration for DynamoDB?

In order to ensure that Amazon OpenSearch Ingestion has the necessary permissions to replicate data across both these systems, the zero-ETL integration feature creates an IAM role with the necessary permissions to read data from Amazon DynamoDB tables and write to an Amazon OpenSearch domain or collection. This role is then assumed by Amazon OpenSearch Ingestion pipelines to ensure that the right security posture is always maintained when moving the data from source to destination.

How can I monitor the state of my integration between Amazon DynamoDB and Amazon OpenSearch Service?

You can view all the metrics related to your zero-ETL integration with Amazon DynamoDB on the dashboards provided by Amazon OpenSearch Ingestion along with real-time logs in Amazon CloudWatch. This enables customers to set up custom alerting that is triggered when user-defined thresholds are breached.

How does Amazon OpenSearch Service zero-ETL integration with Amazon S3 work?

The OpenSearch Service query engine has been rearchitected to support analysis of operational data stored in cloud object stores, such as Amazon S3 and S3-based data lakes. It does so without duplicating data. When seconds make a difference, customers can boost the performance of their queries as well as build fast-loading dashboards by using the new integration’s built-in query acceleration capabilities.

How can I get started using Amazon OpenSearch zero-ETL integration with Amazon S3?

To get started from the AWS Management Console, customers set up a new data source from an existing OpenSearch Service domain running OpenSearch Service version 2.11 or greater. When setting up the new Direct Query data source, customers will need to provide read/write access to Amazon S3 and AWS Glue Data Catalog to facilitate querying data in Amazon S3 from OpenSearch Service. Customers can customize IAM policies to restrict access to specific buckets in Amazon S3 or resources in AWS Glue Data Catalog. After configuring the new data source in the console, customers will go to OpenSearch Service to configure role-based access controls, accelerations to speed up query performance, and optionally out-of-the-box dashboards for popular log type templates such as VPC Flow Logs, Elastic Load Balancer Logs, and NGINX Logs. Customers are billed for the compute resources consumed in the form of Direct Query OpenSearch Compute Units (OCUs; usage rates will apply). After configuring the new data source, customers can start querying their data directly from the OpenSearch API or OpenSearch Dashboards.

How does Amazon OpenSearch Service zero-ETL integration with Amazon S3 pricing work?

Customers only pay for the resources consumed by their workload. OpenSearch Service charges for only the compute needed to directly query external data as well as maintain optional indexes in OpenSearch Service. The compute capacity is measured in OpenSearch Compute Units (OCUs) which is the same units used by Amazon OpenSearch Serverless and Amazon OpenSearch Ingestion. The number of OCUs corresponds directly to the vCPU and memory required to query or maintain indexes based on the data. Customers will see one entry for compute in OCU-hours with the label for direct query. OCUs are billed on an hourly basis with per-minute granularity. If no queries or indexing activities are active, no OCUs are consumed. Costs for Amazon S3 or AWS Glue Data Catalog will be billed separately in the customer’s account. For more pricing details visit the Amazon OpenSearch Service pricing page.

What integration options does Amazon OpenSearch Service have with Amazon Security Lake?

The integration between Amazon OpenSearch Service and Amazon Security Lake provides a streamlined experience for directly searching, gaining insights from, and analyzing data stored in Security Lake, all within the OpenSearch Service. There are two ways you can integrate Security Lake and OpenSearch Service: query-access and data-access.

With the query-access integration, security analysts can use the OpenSearch Dashboards console to run direct queries on the data in Security Lake and leverage on-demand indexing and pre-built content to get started with security-related queries. For the data-access integration, the OpenSearch ingestion service provides a preconfigured Security Lake blueprint, which includes a default configuration for ingesting Open Cybersecurity Schema Framework (OCSF) parquet files from Security Lake. The zero-ETL integration with Security Lake is only offered on collections with OpenSearch UI as opposed to on clusters.

How do I get started with Amazon OpenSearch Service and Security Lake integration?

To get started, you first need to have an existing Security Lake setup in your AWS environment. This will provide the centralized storage and access to your enterprise security data.

You can then enable the integration between the two services through the AWS Management Console or by using AWS CloudFormation templates. This involves configuring the necessary permissions and access controls to allow Amazon OpenSearch Service to securely access and query the data in your Security Lake.

You can then explore the pre-built queries and integrations available through OCSF to quickly get started with common security analytics use cases. You also have the option to configure on-demand indexing of specific data sets from your Security Lake into OpenSearch Service for advanced analytics and visualization needs.

Get started with Amazon OpenSearch Service

Pricing

Learn more about Amazon OpenSearch Service pricing

Visit the pricing page

Console

Ready to build?

Get started with Amazon OpenSearch Service

Amazon OpenSearch Service FAQs

Page topics

General

What is Amazon OpenSearch Service?

Which OpenSearch and Elasticsearch versions does Amazon OpenSearch Service support?

What is an Amazon OpenSearch Service domain?

What does Amazon OpenSearch Service manage on my behalf?

Does Amazon OpenSearch Service support the open-source Elasticsearch and OpenSearch APIs?

What are the Availability Zone (AZ) deployment options available on Amazon OpenSearch Service?

In which regions does Amazon OpenSearch Service offer three AZ deployments?

How can Amazon OpenSearch Service be used for resources running on premises or on other clouds?

Does size flexibility apply to Opensearch Reserved nodes?

Setup and configuration

Can I create and modify my Amazon OpenSearch Service domain through the Amazon OpenSearch Service console?

Does Amazon OpenSearch Service support Amazon VPC?

Can I use CloudFormation Templates to provision Amazon OpenSearch Service domains?

Does Amazon OpenSearch Service support configuring dedicated master nodes?

Can I create multiple Elasticsearch or OpenSearch indices within a single Amazon OpenSearch Service domain?

How do I ingest data into my Amazon OpenSearch Service domain?

Does Amazon OpenSearch Service support integration with Logstash?

Does Amazon OpenSearch Service support integration with Kibana?

What storage options are available with Amazon OpenSearch Service?

What types of EBS volumes does Amazon OpenSearch Service support?

Is there a limit on the amount of storage that can be allocated to an Amazon OpenSearch Service domain?

How are dedicated master instances distributed across AZs?

What is the recommended AZ configuration for production workloads?

How can I configure my domain for three AZ deployment?

Is there a fee for enabling three AZ deployment?

I no longer see the “zone awareness” option in my console. Is my domain no longer zone aware?

How does Amazon OpenSearch Service handle instance failures and AZ disruptions?

If I have only one replica for the indices in my domain, should I use two or three AZs?

How do I leverage three AZ deployment for my VPC domain?

Administration

Can programs running on servers in my own data center access my Amazon OpenSearch Service domains?

How can I migrate data from my existing OpenSearch/Elasticsearch cluster to a new Amazon OpenSearch Service domain?

How can I scale an Amazon OpenSearch Service domain?

Does scaling my Amazon OpenSearch Service domain require downtime?

Does Amazon OpenSearch Service support cross-zone replication?

Does Amazon OpenSearch Service expose any performance metrics through Amazon CloudWatch?

I wish to perform security analysis or operational troubleshooting of my Amazon OpenSearch Service deployment. Can I get a history of all the Amazon OpenSearch Service API calls made on my account?

What is a snapshot?

Why would I need snapshots?

Does Amazon OpenSearch Service provide automated snapshots?

How long are the automated daily hourly snapshots stored by Amazon OpenSearch Service?

Is there a charge for the automated daily hourly snapshots?

Can I create additional snapshots of my Amazon OpenSearch Service domains as needed?

Can snapshots created by the manual snapshot process be used to recover a domain in the event of a failure?

What happens to my snapshots when I delete my Amazon OpenSearch Service domain?

What types of OpenSearch/Elasticsearch logs are exposed by Amazon OpenSearch Service?

What are slow logs?

How can I enable slow logs on Amazon OpenSearch Service?

Can I only enable slow logs for specific indices?

Does turning on slow logs in Amazon OpenSearch Service automatically enable logging for all indexes?

If I turn off the slow logs in Amazon OpenSearch Service, does it mean that log files are no longer being generated?

Can I change the granularity of logging?

Will enabling slow logs or error logs cost me anything?

What kinds of error logs are exposed by Amazon OpenSearch Service?

How can I enable error logs on Amazon OpenSearch Service?

Can I enable error logs for only specific indices?

Are error logs available for all versions of Elasticsearch supported by Amazon OpenSearch Service?

Is there any limit on the size of each log entry?

What is the recommended best practice for using slow logs?

How can I consume logs from CloudWatch Logs?

Are slow logs available for all versions of OpenSearch and Elasticsearch supported by Amazon OpenSearch Service?

Will the cluster have any down time when logging is turned on or off?

Which Elasticsearch and OpenSearch versions does the in-place upgrade feature support?

My domain runs a version of Elasticsearch older than 5.x. How do I upgrade those domains?

Will my domain be offline while the in-place upgrade is in progress?

How can I check if my domain’s Elasticsearch version can be upgraded?

What are the tests done by Amazon OpenSearch Service to validate my domains upgrade eligibility?

Can I update my domain configuration while the version upgrade is in progress?

What happens to the automated system snapshot when the in-place version upgrade is in progress?

How does Amazon OpenSearch Service safeguard against issues that can crop up during version upgrades?

What happens if the system encounters issues while performing the in-place version upgrade?

Can I view the history of upgrades on my domain?

Can I pause or cancel the version upgrade after it has been triggered?

Can I run in-place version upgrade on multiple domains in parallel?

How long does the in-place version upgrade take?

Can I just upgrade the domain quickly without retaining any of the data?

Can I downgrade to previous version if I’m not comfortable with the new version?