Web Scraping With Scrapy and MongoDB – Real Python

Scrapy is a robust Python web scraping framework that can manage requests asynchronously, follow links, and parse site content. To store scraped data, you can use MongoDB, a scalable NoSQL database, that stores data in a JSON-like format. Combining Scrapy...

August 28, 2024

Web Scraping With Scrapy and MongoDB Quiz – Real Python

News

Web Scraping With Scrapy and MongoDB Quiz – Real Python

Interactive Quiz ⋅ 13 QuestionsBy Martin Breuss Share Or copy the link: Copied! Happy Pythoning! In this quiz, you’ll test your understanding of Web Scraping with Scrapy and MongoDB. By working through this quiz, you’ll revisit how to set up...

August 16, 2024

Custom Payroll: MongoDB, Go, Yellowcard Part 3

News

Custom Payroll: MongoDB, Go, Yellowcard Part 3

Prerequisite – check out the second article here Listening to “Strangers” by Lewis Capaldi … I guess it provides a bit of inspiration . As mentioned in the second article of this series, we will connect the dots with helpers,...

August 11, 2024

金融业专家边聊:MongoDB Atlas矢量搜索实际用户案例(搜索内部PDF文档)

News

金融业专家边聊:MongoDB Atlas矢量搜索实际用户案例(搜索内部PDF文档)

用例1: 使用矢量搜索和LLM的PDF搜索应用程序检索增强型生成(RAG) 问题: PDF是非结构化数据难以搜索领先的理赔调查员审查合同和理赔需要高效和准确性来解决这一历史性的繁琐任务总结文件内容指示信息来源挑战: 承保人和理赔调查员要浏览大量的指南、合同和报告页面耗时导致昂贵的错误(风险估算不准确) 详情: 根据用户所在国家提供PDF 矢量搜索查询特定文档过滤掉无关的PDF 索引预筛选: 必须索引具有数据类型的字段只包括具有正确数据类型的记录专用搜索节点架构: 同时支持Atlas搜索和矢量搜索工作负载用例2: 使用LLM和矢量搜索的理赔管理,用于RAG 挑战: 理赔调查员从不同格式的系统中提取和汇总信息内容共享平台客户信息被锁定在遗留的客户关系管理系统中与理赔有关的图片和语音报告非结构化数据简化操作解决方案: Atlas矢量搜索大语言模型(LLM) 检索增强型生成(RAG) 矢量搜索查询: 获取矢量嵌入和纯文本元数据不需要检索数据聚合管道:...

August 5, 2024

Optimizing MongoDB Queries: Best Practices and Tips

News

Optimizing MongoDB Queries: Best Practices and Tips

MongoDB is a popular NoSQL database known for its flexibility and scalability. However, as your dataset grows, poorly optimized queries can lead to performance bottlenecks, slowing down your application and increasing resource consumption. Optimizing your MongoDB queries can significantly enhance...

July 2, 2024

Farewell MongoDB: 5 reasons why you only need PostgreSQL

News

Farewell MongoDB: 5 reasons why you only need PostgreSQL

Discuss the reasons why you should consider PostgreSQL over MongoDB for your next project. In the database world, MongoDB and PostgreSQL are both highly regarded choices. MongoDB, a popular NoSQL database, has gained widespread popularity since its inception in 2009...

July 2, 2024

News

MongoDB Aggregation Pipelines Explained

What is ‘Aggregation’? What is a ‘Pipeline’? What role does they play in MongoDB databases? How are they written? Let’s talk about it in this article from my own perspectives and understanding. Starting as a developer, if you visit the...

July 1, 2024

News

MongoDB Aggregate Functions Guide

Introduction MongoDB, a leading NoSQL database, provides powerful tools for aggregating and transforming data through its aggregation framework. Aggregate functions are crucial for performing operations like filtering, grouping, sorting, and reshaping data. In this blog, we’ll explore the most commonly...

June 30, 2024

Real-time chat communication for a messaging system with NodeJs and Socket.IO

News

Real-time chat communication for a messaging system with NodeJs and Socket.IO

I’m going to be writing about a back-end problem I solved and give an overview of the step-by-step approach I took to solving that problem. Quite recently while working on implementing a messaging system and I faced an issue of...

June 29, 2024

Comprehensive Guide to Schema Inference with MongoDB Spark Connector in PySpark

News

Comprehensive Guide to Schema Inference with MongoDB Spark Connector in PySpark

When working with MongoDB and Apache Spark, you might encounter situations where the MongoDB Spark Connector misses some keys while inferring the schema. This can be due to the connector sampling a subset of documents to determine the schema, potentially...

June 28, 2024

1 2 3